Background
Despite previous studies demonstrating suboptimal appropriate use of stress echocardiography (SE), few interventions have been demonstrated to improve its appropriate use. The aim of this study was to develop a novel mechanism to improve the appropriateness of SE by implementing a point-of-care decision support tool and ordering requisition coupled with an educational strategy.
Methods
A prospective pre- and postintervention analysis was conducted. The intervention included education and the development and implementation of novel ordering requisition coupled with a decision support tool that integrated appropriate use criteria (AUC) for SE.
Results
In the baseline period, 256 consecutive stress echocardiographic studies were evaluated, and 97% were classifiable by the 2011 AUC. During the intervention period, 159 studies were evaluated (98% classifiable). The intervention resulted in an increase in the appropriate proportion from 65% to 76% and a reduction in the rarely appropriate proportion from 31% to 19% ( P = .017). After adjustment for physician specialty, the postintervention period had lower odds of rarely appropriate testing (0.54; 95% CI, 0.3–0.95; P = .04). Cardiology had significant lower odds of rarely appropriate testing (0.23; 95% CI, 0.11–0.50; P < .001) compared with family practice (the reference standard). Vascular surgery had the highest odds (5.76; 95% CI, 2.18–21.52; P = .002) of rarely appropriate testing.
Conclusion
AUC have not previously been applied to SE in a single-payer, publicly funded health system. The development of an educational intervention involving a new requisition and decision support tool that integrated AUC resulted in a significantly reduced proportion of rarely appropriate SE. Cardiologists ordered the highest proportion of appropriate SE. Further study is needed to determine the generalizability of the results.
Highlights
- •
Inappropriate ordering of cardiac diagnostic testing remains an issue.
- •
Little is known about rarely inappropriate stress echocardiography rates in a single-payer health system.
- •
Methods to improve that focus solely through education have shown little success.
- •
A novel approach incorporating education and new ordering requisition (a decision support tool) was implemented.
- •
The 2011 AUC were able to classify the majority of studies.
- •
Rarely appropriate testing decreased by 39% and appropriate testing increased by 17%.
Stress echocardiography (SE) is a cardiac imaging modality that is used to evaluate patients suspected of having coronary artery disease (CAD), valvular heart disease, and other specific cardiac diagnoses. SE is a powerful tool to risk-stratify patients with suspected or proved cardiovascular disease for a wide range of clinical scenarios. As a result of easy access and clear utility, cardiovascular imaging represents one of the fasting growing expenditures on health care budgets. Recently published data demonstrate that adjusted rates of echocardiography use are growing at an annual rate of increase of 5% to 8%.
In response to this temporal trend, the American College of Cardiology Foundation developed appropriate use criteria (AUC) to guide clinicians in determining if echocardiography is indicated in a particular clinical scenario. The AUC for SE were originally developed in 2008 and were most recently updated in 2011, as part of the combined AUC for echocardiography. The most recent AUC allow physicians to classify SE as appropriate, may be appropriate, or rarely appropriate. Previous data suggest not only that the 2011 AUC for SE reliably classify the majority of indications for SE but that appropriate ordering of SE could be improved, with an inappropriate rate of up to 22%.
Most of the previous research on appropriateness has been conducted in multipayer systems, such as in the United States. The proportion of inappropriate SE in a single, government-payer system is unknown. In addition, no interventions using the AUC have been developed and tested specifically for SE. The objectives of the present study were to determine the proportion of inappropriate SE ordered at a large academic health sciences center in a single-payer health system and to develop and test the impact of a novel educational and decision support (DS) strategy on the proportion of rarely appropriate SE ordered.
Methods
Study Design
We conducted a prospective, pre- and postintervention study of an educational intervention aimed at reducing the proportion of inappropriate SE at St. Michael’s Hospital (a tertiary care, 550-bed academic hospital with a large cardiology division) ordered for all patients referred to the echocardiography laboratory (both inpatients and outpatients). Baseline proportions of appropriate, rarely appropriate, and may be appropriate SE were determined prospectively from July 2, 2013, to October 31, 2013 (preintervention). The 2011 AUC for SE were applied to each stress echocardiographic study performed. The cardiologist (either an echocardiography fellow or a staff physician) assigned to SE for the day was responsible for collecting baseline demographics, clinical presentation, previous testing results, and the reason for the study and reporting this information on the data collection form.
When available, the electronic medical record, which includes clinical notes, previous imaging, and laboratory testing, served as an additional resource to supplement the information gathered during the patient encounter. This information was first used to determine if SE was classifiable using the AUC, and if classifiable, determine if it was appropriate, rarely appropriate, or may be appropriate. The terminology used in the 2011 AUC included the previous terms appropriate (scores of 7 to 9), uncertain (scores of 4 to 6), and inappropriate (scores of 1 to 3). This language was changed to reflect the newer terms appropriate (scores of 7 to 9), may be appropriate (scores of 4 to 6), and rarely appropriate (scores of 1 to 3). Once the baseline appropriateness proportion was established, the educational intervention was deployed, and the rarely appropriate proportion was then recorded on a monthly basis after the implementation. The postintervention period lasted from March 1, to May 31, 2014. The St. Michael’s Hospital Research Ethics Board approved the study.
Intervention
The baseline period provided an opportunity to determine who the key ordering providers were. Cardiology, family practice, and vascular surgery were identified, as they ordered the vast majority of stress echocardiographic examinations (92%) and were targeted for the intervention. The educational intervention consisted of two principal components: (1) A 60-min interactive lecture on the AUC for SE given by the principal investigator to the physician-led teams for cardiology, vascular surgery, and the outpatient family practice units and (2) an updated referral form for SE that was designed to provide ordering clinicians with point-of-care DS tool regarding appropriate ordering of SE. The family practice cohort represented a diverse group of physicians, including a specialty “executive” clinic, an academic family practice unit, and community family physicians.
The formal lectures were delivered between November 1, 2013, and January 29, 2014, to the key stakeholder groups. The content included reviewing the AUC for appropriate SE referrals, discussing the local data collected on appropriateness from each stakeholder group obtained during the baseline period, and obtaining feedback on the draft version of the DS tool. The lectures were tailored to be relevant to each stakeholder group and adjusted according to specific feedback. For example, the lecture to vascular surgery emphasized how to risk-stratify patients before high-risk surgery, while the lecture to family practice discussed how and when to conduct stress tests on patients with CAD. In addition, the presentation focused on the local data for each provider and gave feedback about their inappropriate referral rate.
DS Tool and New Ordering Requisition
The updated referral form was created using the preintervention data and the latest AUC for SE. The requisition is a two-sided document. The front page of the requisition provided space to document demographic information. The bottom section of the form listed only appropriate and may be appropriate indications for SE in a color-coded fashion. Rarely appropriate indications were deliberately not listed. Clinicians who wished to order SEs for reasons not listed on the form were required to manually document any additional indication. A free-text space was provided to allow providers to document additional information and/or different indications for SE. No study was refused once the requisition was completed. The DS tool focused on the most common reasons for rarely appropriate testing and provided guidance whether SE was indicated. The choices provided were selected on the basis of feedback from the focus group sessions. The reverse side of the form (DS tool portion) provided an algorithmic approach to help the stakeholders order the correct test for the correct indication. An analysis was performed to determine the most common indications for a rarely appropriate test. The most common rarely appropriate indications were preoperative assessment, ischemia testing in low-risk individuals, and asymptomatic patients. These three indications represented 92% of all inappropriate stress echocardiographic studies. The most common appropriate and may be appropriate indications were emphasized on the DS tool. Iterative changes were made to the form on the basis of the feedback from the lectures. Furthermore, to improve engagement from our key stakeholders, small focus groups were conducted with representatives from the key groups (three family physicians, one vascular surgeon, and one cardiologist). This process helped us obtain specific feedback on how to improve the DS tool before deployment. Performance of SE was contingent on completion of the new form, and therefore compliance with filling the form was expected to be 100%. The final version was launched on February 12, 2014, and is presented in Figure 1 .
Classification of SE Appropriateness
All the available clinical information collected during the patient interview, as well as the clinical information taken from the electronic medical record (when available), was used to assign the appropriateness of SE. The indication documented by the practitioner on the ordering requisition was not used to adjudicate the appropriateness of SE but only to understand the reason for the test. The assessment of the appropriateness of SE was made by two blinded level III echocardiographers. After deliberation between the two cardiologists, final designation of appropriateness was assigned.
Outcomes
The primary outcome was the proportion of rarely appropriate SE ordered as determined by the AUC before and after the intervention. The monthly rarely appropriate proportions were tracked throughout the period of study. An overall proportion and a proportion from each subspecialty were measured during the pre- and postintervention periods. Secondary outcomes included the overall appropriate and may be appropriate proportions of SE according to the AUC. Additional measures were the number of stress echocardiographic studies with positive results stratified by appropriateness (i.e., appropriate, may be appropriate, and rarely appropriate). As a process measure, the percentage of data collection forms completed as a proportion of total stress echocardiographic studies performed was determined. This information was used to demonstrate that the majority of the studies performed were captured within the data set.
To ensure that the total number of stress echocardiographic studies performed monthly was not reduced by the intervention, total studies performed by month were documented before and after the proposed intervention.
Statistical Analysis
Categorical variables for ordering characteristics, patient demographics, and appropriateness ratings using the 2011 AUC were compared using χ 2 or Fisher exact tests as appropriate. Continuous variables are reported as mean ± SD. Statistical significance was indicated by a two-tailed P value < .05. Odds ratios together with 95% CIs and P values are reported from the generalized linear mixed model.
To assess provider group variation, a generalized linear mixed model that accounted for within-physician correlation and also adjusted for physician specialty was constructed to assess the risk for inappropriate testing by provider. Family practice was arbitrarily chosen as the reference group.
Results
Study Cohort
There were 256 consecutive studies in the preintervention cohort and 159 consecutive studies in the postintervention cohort. Table 1 lists the patient-specific characteristics of each cohort. The mean age of the preintervention cohort was 62.3 ± 12.7 years, and the mean age of the postintervention cohort was 61.8 ± 13.4 years. The majority of patients were men (62% and 65% in the pre- and postintervention cohorts, respectively). Cardiologists ordered the largest number of stress echocardiographic studies in both the pre- and postintervention cohorts, followed by family practitioners and vascular surgeons. A minority of studies were ordered by anesthesiologists and internal medicine physicians. There were no statistically significant differences in pre- and postintervention patient characteristics with respect to percentages with diabetes, hypertension, hypercholesterolemia, smoking histories, family histories of premature CAD, previous myocardial infarctions, previous angioplasty, history of heart failure or recent hospitalization, previous coronary artery bypass surgery, or valvular heart disease. More patients had previous noninvasive stress testing (within the past year) in the preintervention cohort compared with the postintervention cohort (26% vs 16%, P = .025). The most common type of previous stress test was an exercise electrocardiographic treadmill test (72% and 80% in the pre- and postintervention cohorts, P = .92). The vast majority of patients had interpretable electrocardiograms obtained before SE (95% and 92% in the pre- and postintervention cohorts, P = .28) and could exercise more than four metabolic equivalents (82% and 79% in the pre- and postintervention cohorts, P = .27). Almost all of the studies were classifiable (97% and 98% in the pre- and postintervention cohorts, P = .75) by the AUC. During the pre-ntervention period, there was a 98% completion rate of the data collection form, and in the postintervention period, 93% of the forms were completed ( P = .02).
| Variable | Preintervention ( n = 256) | Postintervention ( n = 159) | P |
|---|---|---|---|
| Age (y) | 62.3 ± 12.7 | 61.8 ± 13.4 | .75 |
| Men | 62% (158) | 65% (103) | .53 |
| Specialty | .03 ∗ | ||
| Anesthesia | 4% (9) | 3% (5) | |
| Cardiology | 49% (126) | 63% (100) | |
| Family practice | 29% (75) | 17% (27) | |
| Internal medicine | 4% (9) | 2% (3) | |
| Vascular surgery | 14% (37) | 15% (24) | |
| DM | 21% (54) | 19% (31) | .70 |
| HTN | 52% (134) | 53% (84) | .92 |
| Hyperlipidemia | 55% (141) | 55% (88) | .96 |
| Smokers (current or quit <6 mo ago) | 18% (47) | 18% (28) | .85 |
| Family history of CAD | 17% (44) | 23% (37) | .13 |
| History of MI | 14% (35) | 14% (22) | .96 |
| History of PCI | 9% (22) | 14% (22) | .09 |
| Valvular HD | 5% (12) | 7% (11) | .33 |
| CABG | 7% (18) | 6% (10) | .77 |
| HF | 2% (6) | 1% (1) | .26 ∗ |
| Recent hospitalization (<6 mo) | 3% (7) | 3% (4) | 1.00 ∗ |
| Previous stress test (within past year) | 26% (66) | 16% (26) | .03 |
| Type | .92 ∗ | ||
| GXT | 72% (47) | 80% (20) | |
| MIBI | 24% (16) | 20% (5) | |
| Dobutamine SE | 2% (1) | 0% | |
| ESE | 2% (1) | 0% | |
| Interpretable ECG at start of SE | 95% (242) | 92% (146) | .28 |
| Exercise >4 METs | 82% (211) | 79% (126) | .27 |
| Studies classifiable by AUC | 97% (249) | 98% (156) | .75 |
| Studies with DCF filled out | 98% | 93% | .02 |
Primary Analysis
As shown in Table 2 , compared with the preintervention period, the proportion of rarely appropriate SE was lower in the postintervention group (19% vs 31% in the preintervention group), and the proportion of appropriate SE was higher (76% vs 65% in the preintervention group) (overall P = .017). There was no significant change in the proportion of may be appropriate SE. There was no statistically significant difference between physician subgroups in the proportion of rarely appropriate studies ordered ( Table 2 ). Figure 2 demonstrates the rarely appropriate proportion as a run chart depicting the impact of the various interventions on ordering appropriateness over time.
