PERFORMANCE MEASURES USING ELECTRONIC HEALTH RECORDS:
FIVE CASE STUDIES
Jinnet Briggs Fowles, Elizabeth A. Kind, and Shadi Awwad Park Nicollet Institute
Jonathan P. Weiner and Kitty S. Chan Johns Hopkins Bloomberg School of Public Health
Patricia J. Coon, Billings Clinic; James T. Krizak and Lynne Dancha, HealthPartners; Nancy Jarvis, Park Nicollet Health Services; Dean F. Sittig and Brian L. Hazlehurst, Kaiser Permanente of the Northwest; and Mark J. Selna, Geisinger Health System
ABSTRACT: This report examines the experiences of five provider organizations in developing, testing, and implementing quality-of-care indicators, based on data collected from their electronic health record (EHR) systems. HealthPartners used the EHR to compile blood pressure measurements, Park Nicollet Health Services developed a composite measure for care of people with diabetes, Billings Clinic tested an automatic alert on potential interactions between antibiotics and the anticoagulant warfarin, Kaiser Permanente used a natural-language processing tool for counseling about tobacco use, and Geisinger Health System explored ways of reconciling Problem Lists and provider-visit notes regarding high-impact chronic-disease diagnoses. Common themes emerged from these case studies. They included challenges—of ensuring the validity and reliability of data, efficient workflow, and staff support—but the providers’ successes in implementing their respective EHR-based quality measures demonstrated that such measures are adaptable to different EHR systems, amenable to improvement, and worth pursuing. Support for this research was provided by The Commonwealth Fund and The Robert Wood Johnson Foundation, with additional support from the Agency for Healthcare Research and Quality. The views presented here are those of the authors and do not necessarily reflect those of the two sponsors or their directors, officers, or staff. This document and other Commonwealth Fund publications are available online at www.commonwealthfund.org. To learn more about new publications when they become available, visit the Fund’s Web site and register to receive e-mail alerts. Commonwealth Fund pub. no. 1132.
CONTENTS List of Figures and Tables…………………………………………………………………………………….. iv
About the Authors………………………………………………………………………………………………….v
Executive Summary …………………………………………………………………………………………….. vi
Case Study #1 Using the EHR to Compile Blood Pressure Measurements ……………………6
Case Study #2 Optimal Care for Diabetes: Linking Measurement with Improvement …….12
Case Study #3 Warfarin/Antibiotic Rule for the EHR……………………………………………….19
Case Study #4 Assessing Clinician Adherence to Smoking-Cessation Guidelines Using MediClass: An Innovative Natural-Language Processing Tool……………………24
Case Study #5 Problem List Diagnosis Reconciliation ……………………………………………..29
Five Case Studies: Conclusions ……………………………………………………………………………..34
LIST OF FIGURES AND TABLES Figure 1 HealthPartners EMR Data in Contrast with
HEDIS Controlling High Blood Pressure Results …………………………………….10
Table 1 Results for Electronic Health Records (EHRs) and Blood Pressure (BP) Readings……………………………………………………………….11
Figure 2 Extended Laboratory Menu (ELM) vs. NON-ELM Percent Patients Grand Slam……………………………………………………………………………..16
Table 2 The “5As” Recommended by the Current U.S. Public Health Service Clinical Practice Guideline for Tobacco Treatment and Prevention ……………25
Table 3 Diagnosis Specificity for Selected Chronic Diseases— Problem List vs. Encounters ………………………………………………………………….33
ABOUT THE AUTHORS Jinnet Briggs Fowles, Ph.D., is a senior vice president at the Park Nicollet Institute, where she has worked since 1985. Her areas of expertise include quality measurement, data reliability, and consumer and provider responses to information about quality. She earned her M.S. in Library Science from Simmons College and her Ph.D. from Stanford University in Communications Research. Jonathan P. Weiner, Dr.P.H., is a professor of health policy and management at the Johns Hopkins Bloomberg School of Public Health. He is also deputy director of the Johns Hopkins Health Services R&D Center. His areas of expertise include primary care, quality of care, risk adjustment and the application of electronic health records, claims, and other computerized data to health services research and evaluation. He holds a doctorate in health services research from Johns Hopkins. Kitty S. Chan, Ph.D., is assistant professor in the Department of Health Policy and Management at the Johns Hopkins Bloomberg School of Public Health. Her areas of expertise include psychometrics and quality measurement. She received her Ph.D. in Health Services Research from Johns Hopkins in 2001. Prior to joining the faculty at Johns Hopkins in 2004, she was associate policy researcher at the RAND Corporation between 2001 and 2004. Elizabeth A. Kind M.S., R.N., is director of survey research at the Park Nicollet Institute’s Health Research Center where she has worked since 1983. Her responsibilities include project management and overseeing instrument design and field data collection for the Center’s research. She earned her master of science degree from the University of Minnesota. Shadi Awwad, M.S., at the time of this study was a graduate assistant at the Park Nicollet Institute, completing the University of Minnesota’s master’s program in Health Services Research and Policy. Mr. Awwad is currently assistant to the Secretary General, Royal Hashemite Court, Jordan, Asia.
The emergence of the electronic health record (EHR), also termed the electronic medical record, has made new indicators of quality and safety both necessary and feasible. By developing appropriate indicators now, we can integrate them into evolving EHR systems early on rather than try to add them after the fact—a much more difficult task. This report examines the experiences of five provider organizations in developing, testing, and implementing such indicators, based on data collected from their EHR systems.
To set the stage, we developed a typology for categorizing electronic measures (“e-indicators”) of quality and safety, with special reference to ambulatory care. The five categories are:
1. Translational e-indicators are measures that have been translated from existing—“traditional”—measurement sets (e.g., HEDIS or NQF standard measures) for use in health information technology (HIT) platforms.
2. HIT-facilitated e-indicators are measures that, while not conceptually limited to HIT-derived data sources, would not be operationally feasible in settings without HIT platforms. Measuring clinical physiologic outcomes on 100 percent of patients, for instance, would not be amenable to traditional systems.
3. HIT-enabled e-indicators are innovative measures that would not generally be possible outside of the HIT context. These indicators are linked to unique HIT capabilities such as computerized provider order entry (CPOE), clinical decision support systems (CDSS), biometric devices, or Web-based patient portals.
4. HIT-system-management e-indicators are measures needed to implement, manage, evaluate, and generally improve HIT systems. They are primarily intended for use by the parent organization.
5. “E-iatrogenesis” e-indicators are measures of patient harm caused at least in part by the application of health information technology. They assess the degree to which unanticipated quality and safety problems arise, whether of human (provider or patient), technical, or organizational/system origin.
The case studies presented in this report illustrate the use of the first four categories
of e-indicators. The HealthPartners case study analyzed the potential of EHRs to compute traditional quality measures (in this case, blood pressure control) aimed at reducing the time and cost required to assemble the data. The Park Nicollet Health Services case
study illustrated the power of the EHR to assemble composite measures (in this case, diabetes) that are theoretically possible without an EHR but infeasible in practice. The Billings Clinic case study exemplified the HIT strengths of EHRs to coordinate care and measure its outcomes, in this case for a warfarin/antibiotic alert tied to a warfarin clinic. The Kaiser Permanente of the Northwest case study overcame the “free-text dilemma”—that free text, or unstructured information, cannot be readily used for quantitative analyses—by using natural language processing to capture information in text notes. Work at Geisinger Health System, meanwhile, focused on reconciling information on the health problem list (a structured-text field) with information in the visit note (an unstructured-text field).
From these case studies, a number of common themes emerged:
• It is striking how much more clinically relevant measures can become when they are HIT-based. For example, a composite measure reflects a more complete clinical picture of a person with diabetes than a single component of the measure can.
• A major barrier in conceptualizing and developing the e-indicators was the validity of EHR-extracted data, which critically depends on use of the correct patient population. If patients are incorrectly included in or excluded from a measure, the quality measures will be inaccurate.
• Another major barrier was the sometimes questionable reliability of EHR- extracted data, particularly when their collection and recording were inconsistent; the case studies suggested that it was difficult to consistently code data about patients, diagnoses, and procedures. But in addition to identifying these accuracy concerns, most of the case studies implemented workable solutions.
• Prior to implementation of the measures, providers expressed concern that EHRs would hinder workflow or suffer from staff resistance. Surprisingly, these issues did not present themselves. To the contrary, the case studies indicated that EHR systems enhanced workflow by automating key communications between staff and improving patient-record accessibility across different clinics.
• Measures that translated established quality indicators had the easiest transition into EHR implementation. Measures incorporating or evaluating HIT-specific features, such as automated alerts and free-text analysis, tended to be specialized to particular systems and not so easily incorporated into other systems. Nonetheless, most of the providers were confident that the concepts could be adapted to different EHR system types and that virtually all of the problems
encountered were amenable to performance improvement—often made possible by the EHR.
The success of these providers in implementing EHR-based quality measures
demonstrates that such measures are worth pursuing, despite the challenges of ensuring the validity and reliability of data, efficient workflow, and staff support.
PERFORMANCE MEASURES USING ELECTRONIC HEALTH RECORDS:
FIVE CASE STUDIES
Electronic health records (EHRs), also termed electronic medical records, are not yet widespread in the United States, though several leading provider organizations have begun to deploy them, and they are widely considered the wave of the future. EHRs and related health information technology (HIT)—such as computerized physician order entry (CPOE), clinical decision support systems (CDSS), and Web-based patient portals—significantly enhance our ability to evaluate the processes and outcomes of health care and the degree to which consumer needs are being met. But these and other traits of EHRs call for new indicators of quality and safety. By developing appropriate indicators now, we can integrate them into evolving EHR systems early on rather than try to add them after the fact—a much more difficult task.
This report examines the experiences of five provider organizations in
developing, testing, and implementing quality-of-care indicators, based on data collected from their EHR systems. While the focus of each case study was unique, they presented common strengths and weaknesses.
EHR systems should have the following features:
• Accuracy (validity). Data derived from patient records, both inside and outside the system, must be correct and complete.
• Standardization (reliability). Information taken from each patient must be standard and consistent, both across the subject population and within individual patient records.
• Generalizability. Measures should be able to be translated effectively between different data-collection systems.
• Workflow efficiency. Data collection should be structured as part of the work process to avoid negatively affecting patient visits—and ideally, to make visits more efficient.
• Staff support. Management and staff should together develop an infrastructure that encourages support of and adherence to the quality-measurement program.
These characteristics embrace the continuum of issues encountered—technical
issues (such as validity, reliability, generalizability), human-technology interface issues (workflow efficiency), and cultural issues (staff support)—when using EHRs to support quality measurement.
Quality indicators have traditionally been developed by organizations such as the
National Committee on Quality Assurance and the American Medical Association’s Physician Consortium for Performance Improvement. Developed measures have been approved for use by organizations such as the National Quality Forum and the Joint Commission for the Accreditation of Healthcare Organizations. Still other organizations, such as the Ambulatory Quality Alliance, Bridges to Excellence, and Medicare, are primarily responsible for implementing the measures.
These indicators traditionally depend on information derived from insurance
claims (or other billing data), medical records, and quality-of-care surveys of patients. As many leading provider organizations have implemented large EHR systems, quality information has also increasingly come from the EHRs themselves.
EHRs and related HIT components offer many opportunities for organizations
wishing to dramatically expand their quality- and safety-improvement activities or other types of performance monitoring. However, because the deployment of EHRs can also have some negative consequences, a number of related quality challenges are beginning to emerge. Often, they arise during the transition from manual to electronic systems (Ash 2004, Bates 2001, Berger 2004). But even after a system has become functional, problems can persist (Koppel 2005).
We have developed a typology for categorizing the broad types of electronic
indicators (“e-indicators”) of quality and safety, with special reference to ambulatory care. It was based on our understanding of how interoperable EHRs and other types of HIT have been used to date, as well as on the potential that we, the members of our provider consortium, and the early adopters we interviewed envision they may have before long.
This typology is not meant to replace other ways of thinking about quality—for
example, the Institute of Medicine’s six attributes of quality (safe, effective, patient-
centered, timely, efficient, and equitable) or Donabedian’s triumvirate of structure, process, and outcome. A new typology of e-indicators is necessary, given the functions and capabilities not seen before and the system-induced problems that have emerged, and we believe the typology can be used as an adjunct. In that way, it can help us communicate with one another as HIT-based quality improvement and monitoring systems are developed, implemented, and evaluated.
The five-class typology of electronic indicators we propose is as follows:
1. Translational e-indicators are measures that have been translated from
existing—“traditional”—measurement sets (e.g., HEDIS or NQF standard measures) for use in HIT platforms.
Examples of translational measures include, for example, the number of patients with diabetes having an eye-care referral or the number of children receiving appropriate immunizations. Issues that surround the comparability of such translational measures have been the subject of a number of recent papers (Tang 2007).
If a traditional paper-chart-derived measure (e.g., blood-pressure readings or actual laboratory values) is expanded from a limited subsample of patients to a full patient population, it should be considered an “HIT-facilitated” measure (see below) rather than a translational measure.
2. HIT-facilitated e-indicators are measures that, while not conceptually limited to HIT-derived data sources, would not be operationally feasible in settings without HIT platforms.
Examples of HIT-facilitated measure include: clinical outcomes of 100 percent of patients based on physiologic measures such as Body Mass Index, blood pressure, or laboratory values; percentage of cases in which clinicians or patients receive reminders for preventive screening; and percentage of newly written prescriptions that are filled by the patient within seven days.
3. HIT-enabled e-indicators are innovative measures that would not generally be possible outside of the HIT context. These indicators, linked to unique HIT capabilities such as CPOE, CDSS, biometric devices, or Web-based patient portals, may involve only one HIT component or require the interaction of several.
Examples of HIT-enabled measures include: percentage of patients for whom real-time CDSS modules have been appropriately applied in support of the diagnostic process; percentage of congestive-heart-failure patients with daily e-monitored weight gain greater than x pounds, acted upon by the responsible
clinician within y hours; percentage of patients who respond appropriately to messages regarding abnormal test results; percentage of generalists who view the medical notes added by a consulting specialist within seven days of the consult; and percentage of overweight adults referred to nutrition class through a CPOE.
4. HIT-system-management e-indicators are measures needed to implement, manage, evaluate, and generally improve HIT systems, and they are primarily intended for use by the parent organization. However, these measures can also be used by an external body (e.g., a payer) to evaluate the organization and the HIT system it deploys for the consumer population of interest.
Examples of HIT-system-management measures include: EHR item-completion rates; attainment of community interoperability targets; a CDSS algorithm’s accuracy; percentage of real-time alerts bypassed by the clinician; percentage of patient-allergy lists reviewed by patients (via Web portal) annually; and ease of access of measures that are maintained in the free-text section of the EHR.
5. “E-iatrogenesis” e-indicators are measures of patient harm caused at least in part by the application of health information technology. They assess the degree to which unanticipated quality and safety problems arise, whether of human (provider or patient), technological, or organizational/system origin, and they may involve errors of commission or omission (Ash 2004, Campbell 2006, Weiner 2007).
Examples of e-iatrogenesis measures include: percentage of patients receiving incorrect medications or procedures because of HIT-related errors in the CPOE process; percentage of patients experiencing a degree of harm from an unanticipated care-delivery event; percentage of significant CDSS errors (either type 1 or type 2) for a particular condition; human/machine interaction errors that lead to an incorrectly entered diagnosis; and the number of patients experiencing harm because they received another patient’s orders by CPOE.
This measure is designated separately because of its key importance to those concerned with quality and safety in the HIT context. The case studies of this report illustrate the use of the first four categories of e-
indicators: translational (HealthPartners), HIT-facilitated (Park Nicollet), HIT-enabled (Billings Clinic), and HIT-system-management (Geisinger, Kaiser-Portland).
• The HealthPartners case study analyzed the potential of EHRs to compute traditional quality measures (in this case, blood pressure control) aimed at reducing the time needed to assemble the data.
• The Park Nicollet case study illustrated the power of EHRs to assemble composite measures (in this case, diabetes) that are theoretically possible without an EHR but infeasible in practice.
• The Billings Clinic case study exemplified the HIT strengths of EHRs to coordinate care and measure its outcomes, in this case for a warfarin/antibiotic alert tied to a warfarin clinic.
• The Kaiser-Portland case study overcame the “free-text dilemma”—that free text, or unstructured information, cannot be readily used for quantitative analyses. They used natural-language processing, in this case for counseling about tobacco use, to capture information in text notes.
• Work at Geisinger focused on reconciling information on the health-problem list (a structured-text field) with information in the visit note (an unstructured-text field).
These case studies, involving five leading provider organizations, highlighted
the variety, strengths, and potential challenges of quality indicators associated with EHR data. Although the particulars of each case study were different, the cases presented many common benefits and barriers. Thus, the overall lessons from these providers’ experiences can help guide future efforts to integrate quality measurement into EHR systems.
CASE STUDY #1 Using the EHR to Compile Blood Pressure Measurements
James T. Krizak, Shadi Awwad, and Lynne Dancha Introduction
Hypertension Controlling high blood pressure (CBP), an important clinical outcome, is now a quality- of-care measure adopted by the National Committee for Quality Assurance (NCQA) for the Health Plan Employer Data and Information Set (HEDIS). NCQA added this measure to its core measurements because hypertension has been a challenge for providers to control—through the mid-1990s, only 27.4 percent of patients with hypertension had their blood pressure under control. Because CBP is included in HEDIS accreditation measures, health plans are focusing more attention on improving this rate.
CBP has also been adopted by Minnesota Community Measurement, a nonprofit entity whose mission is to accelerate the improvement of health by publicly reporting health care information. While NCQA publishes HEDIS measurement results for numerous national health plans, Minnesota Community Measurement publishes quality results for medical groups in Minnesota. HealthPartners HealthPartners is the largest consumer-governed nonprofit health care organization in the nation; with 635,000 members, a full-service hospital, more than 50 clinical sites, and nearly 580 practicing physicians.
HealthPartners’ own Medical Group, which treats some 40 percent of the health plan’s members, fully converted to electronic health records (using the EPIC system) by January 2005. While other medical groups that treat members of HealthPartners have also converted to electronic health records, HealthPartners currently does not have access to their data. Thus our study population represented only 40 percent of the people enrolled in the health plan.
HealthPartners has played major leadership roles in the development both of HEDIS and Minnesota Community Measurement®. While it is now producing numerous quality measures using EHR data—such as for asthma, depression, diabetes, hypertension, children’s health, and adult preventive services—this case study focused on the CBP measure. Goal By following HEDIS specifications, a health plan is assured that its results will be accepted by NCQA and also that the results can figure in comparisons with other providers that use HEDIS specifications. Within NCQA’s Quality Compass grade system, there are five bands for any HEDIS measure, corresponding to a health plan’s compliance percentage rate. After achieving certain band grades, a health plan may receive accreditation from NCQA, thereby greatly increasing its credibility in the community. Because blood pressure control is a key indicator among HEDIS measurements, it is a prerequisite to accreditation, and, to gain that status, HealthPartners’ goal is to reach CBP’s highest band. Sample HEDIS guidelines require that the population used for CBP include only patients who are 46-85 years old as of December 31 of the measurement year. A subject is considered to have hypertension if there is at least one outpatient encounter, verified by chart review, with an ICD-9 (International Classification of Disease version 9) code of 401 (Essential Hypertension) during the first six months of the measurement year. The patient’s blood pressure is considered to be under control when his or her reading is less than or equal to 140/90. Measurement The actual measurement is the percentage of hypertensive patients satisfying the criteria of the above paragraph. Historically, collection of data for the CBP measure required an electronic scan of administrative claims data for an ICD-9 diagnosis code of hypertension, followed by a manual review of the identified patients’ medical records to verify diagnosis and record systolic and diastolic blood pressure readings.
In this case study, HealthPartners followed the HEDIS specifications for the CBP measure—which dictate the minimum size of the random sample—in order to determine the measurement population. The health plan then increased the sample size by 10 percent to allow for diagnostic coding issues on claims data (occasionally, patients without hypertension or with an inappropriate age are identified and need to be removed
from the sample). For 2005 dates of service, HealthPartners’ sample for the blood pressure measure was 1,121 patients, identified by looking for a diagnosis of hypertension among claims data.
NCQA requires that the hypertension diagnosis be verified by chart review, so
HealthPartners adapted the EHR to do this verification. An electronic scan of structured analytic fields in HealthPartners EHR data found that 464 patients (41%) in the sample population had a substantiated diagnosis of hypertension and a subsequent blood pressure reading less than or equal to 140/90. The remaining 657 patients required a manual review of patient medical records to substantiate the diagnosis of hypertension or to find that blood pressure was under control. These 657 patients either were not HealthPartners Medical Group patients (so they had no HealthPartners’ EHR) or they were HealthPartners members whose blood pressure was not under control.
A benefit of using EHR data was elimination of the need for manual review of
464 patient medical records, which saved HealthPartners about $5,500 (based on a calculated average cost of $12.00 per record to review). Although such savings per individual are small, when applied over a large population of patients—and especially when combined with other indicators that use EHR data to achieve standard levels—the savings add up. Verification and Reliability Using EHR databases in the data warehouse, 464 patients were identified as having hypertension but having their blood pressure under control. To test the data’s reliability, we examined the records of a random sample of these patients. Individual records were manually reviewed, using the EHR’s front end—the section that auditors use to confirm the diagnosis or find the current blood pressure. Similarly, the date of the most recent blood pressure test and the blood pressure levels recorded from the EHR database were verified. This test was repeated three times because the first and second reviews uncovered errors in the logic that inappropriately contradicted the EHR warehouse. After correcting the logic, a 100 percent match was achieved between records retrieved from the warehouse and the way they appeared in the EHR front end. Results HEDIS guidelines have been followed since 2000, with results showing a positive trend in the CBP measure, as shown in the table. The rate represents the percentage of commercial members identified as having both hypertension and their blood pressure under control (≤140/90).
In HEDIS year 2006, HealthPartners was 0.8 percent away from achieving Band 1—the highest rank in the NCQA ratings, representing the 90th percentile of all HEDIS managed care organizations. The best practice of providers for the CBP measure was CIGNA HealthCare of Colorado, Inc., at 83.7 percent.
Between 2002 and 2003, there was a large jump in the CBP rate, which analysis
correlated with HealthPartners’ full implementation of its EHR system. During that period the number of EPIC (warehouse that stores EHR data) members increased by 75.8 percent and the number of blood-pressure readings taken from members and stored in EPIC went up 186.7 percent. The table shows these variables’ values from services years 2000 to 2006.
Possible inferences from these data:
• Implementation of the EHR results in better documentation, which then leads to an increase in people with hypertension having their blood pressure taken, recorded, and managed, which can then lead to more patients being in control.
• The EHR certainly makes it much easier to find data. Before, auditors had to
manually chase down a patient’s data, with the possibility of never finding what they were looking for. Now, a query can be done against the EPIC warehouse, and all blood pressures are identified, no matter which HealthPartners clinic was visited. This ready availability of data can lead to a better compliance rate.
Challenges When EHR data were accessed for the first time to report CBP, lack of familiarity with the new warehouse made finding the needed data elements difficult to achieve. EHR extracts were pulled from EPIC and reviewed to verify that the correct codes were being used for the hypertension confirmation and that the blood pressure readings were complete. Still, because review of the EPIC warehouse is an ongoing process, accurate results cannot yet be guaranteed. Conclusions The hopes for this movement to EHR are to eliminate the time and cost of pulling data manually and to make the results more robust. When using EHR data, multiple indicators—such as blood pressure, LDL, HDL, HBA1c, BMI, immunization history, and smoking status—can be pulled and reviewed simultaneously. Also, it seems that with the implementation of an EHR system, HEDIS compliance rates may actually increase, as
they appear to do for the Controlling High Blood Pressure measure (see figure below). (The apparent drop in performance in 2006 was, in fact, due to a HEDIS specification change. The requirement for being under control went from <140/90 to <140/90, resulting in a surprising impact on this measure’s rates.)
The post PERFORMANCE MEASURES USING ELECTRONIC HEALTH RECORDS: appeared first on Infinite Essays.
Hi there! Click one of our representatives below and we will get back to you as soon as possible.