Comparison of the 2007 and 2011 Appropriate Use Criteria for Transthoracic Echocardiography in Various Clinical Settings




Background


The 2007 Appropriate Use Criteria (AUC) for echocardiography was revised in 2011 to cover a wider range of scenarios of use. Previous studies of the 2007 AUC found a relatively large number of unclassified transthoracic echocardiograms (TTEs).


Methods


We conducted a retrospective chart review comparing TTE usage in three clinical environments: academic inpatient, academic outpatient, and community outpatient. We assessed the TTE ordering behavior using both the 2007 and 2011 AUC.


Results


We reviewed 150 consecutive TTEs performed in 2011 in each of the three practice settings (total 450). Using the 2007 AUC, 347 TTEs (77%) were classifiable, and, using the 2011 AUC, 441 TTEs (98%) were classifiable ( P < .001). Of the classified studies, the percentage of appropriate TTEs using the 2007 AUC was 83% and using the 2011 AUC was 71% ( P < .001). Using the 2007 and 2011 AUC, the percentage of inappropriate TTEs was 17% and 22% ( P = .14), and the percentage of uncertain TTEs was 0% and 7% ( P < .001), respectively. The rate of inappropriate studies was greatest in the outpatient academic medical center (30%), followed by the outpatient community health center (21%) and the inpatient academic medical center (14%; P = .004).


Conclusions


The 2011 AUC classified a significantly greater proportion of TTEs than the 2007 AUC across a variety of practice settings. The rate of appropriate TTEs was lower using the 2011 AUC, and the appropriateness ratings varied by clinical setting. These findings suggest that the expanded AUC offer an opportunity for improvement in TTE usage.


Transthoracic echocardiography (TTE) is an important tool in the diagnosis and management of cardiovascular disease, but concerns have been raised regarding its usage and growth. To provide guidance for the appropriate ordering of TTE, the American College of Cardiology Foundation, American Society of Echocardiography, and other subspecialty societies published appropriate use criteria (AUC) for TTE in 2007. Awareness of, and adherence to, the AUC has become an important focus for quality improvement in echocardiography laboratories, and tracking appropriateness has become a requirement for accreditation.


Implementation studies examining the 2007 AUC for TTE have been conducted in different clinical settings. These studies have reported that a large majority of TTEs (80–90%) are ordered for appropriate indications. The percentage of appropriate TTEs has been greater in inpatient than in outpatient settings. Despite these encouraging results, 11–35% of TTEs in these implementation studies have not been classifiable using the 2007 AUC. Given the relatively high unclassifiable rate and the evolving use of TTE in clinical practice, the AUC for TTE were revised and expanded in 2011. Several studies have analyzed the 2011 AUC ; however, to our knowledge, no studies to date have evaluated the 2011 AUC across various practice settings, including an outpatient community practice setting.


To assess the performance of the 2011 AUC for TTE, we applied both the 2007 and 2011 AUC to TTE ordering practices in three distinct clinical environments: inpatients at an academic medical center (IAMC), outpatients at an academic medical center (OAMC), and outpatients at a community health center (OCHC). We hypothesized that the 2011 AUC would classify a greater number of TTEs than would the 2007 AUC and that this would affect the calculated rate of appropriate TTEs.


Methods


Study Population


We conducted a retrospective chart review of 150 consecutive TTEs performed in each of the three clinical settings starting on April 1, 2011. The three settings were the Inpatient Echocardiography Laboratory at Massachusetts General Hospital (IAMC), Outpatient Echocardiography Laboratory at Massachusetts General Hospital (OAMC), and the Revere Community Health Center (OCHC). The OCHC included primary care physicians and was reflective of a community-based practice. The exclusion criteria included patient age younger than 18 years, TTEs ordered as a part of a research study, insufficient information in the electronic medical record (EMR) preventing determination of the clinical reason for ordering the TTE, and access to the EMR was restricted owing to privacy restrictions.


The Partners Healthcare institutional review board reviewed and approved the study protocol. We had no contact with the patients nor did we attempt to influence the care of the patients in any way.


Data Collection


The ordering information, patient demographics, and TTE results were determined from a review of the EMR. The signs and symptoms and reasons for ordering TTE were abstracted from the TTE order and EMR. TTE ordering is performed electronically for inpatients and by paper order for outpatients at the academic and community sites. The inpatient ordering system contains an electronic prompt regarding the date of a patient’s most recent TTE; otherwise, no decision support tool is available. The EMR is a comprehensive database that includes physician notes, imaging results, and laboratory findings, and captures the clinical activities of all three practice settings. The date of each patient’s most recent previous TTE, if any, was also noted. The EMR contains the TTE reports from all hospitals within our healthcare network. Typically, other TTE results are scanned into the system. Three study investigators (R.S.B., D.M.C., R.B.W.) reviewed the TTEs. They classified the indication for each TTE using the 2007 and 2011 AUC, allowing for determination of an appropriateness rating (appropriate, uncertain, or inappropriate). If the reason for the TTE did not have a corresponding indication in the 2007 or 2011 AUC, it was considered unclassifiable. At least 2 of the investigators reviewed each TTE order, and agreement on the AUC classification was required. For TTEs in which an initial consensus was not reached, a review of the TTE order and EMR was conducted by a third investigator and a consensus reached. The need for review by all 3 investigators was required for 15 TTEs. If a patient had undergone multiple TTEs during the study period, each TTE was considered independently and included in the analysis.


The patient demographics recorded were as follows: age, gender, cardiac risk factors (diabetes mellitus, smoking history, hypertension, and hyperlipidemia), previous cardiac procedures (coronary artery bypass grafting, valve surgery, and percutaneous coronary intervention), comorbidities (angina, heart failure, previous myocardial infarction, atrial fibrillation, peripheral arterial disease, chronic obstructive pulmonary disease, chronic kidney disease, connective tissue disease, dementia, and cancer), hospitalization within the previous 6 months, and results of other cardiac imaging (cardiac computed tomography, cardiac magnetic resonance imaging, and single photon emission computed tomography) performed within the previous year.


Echocardiography


Complete two-dimensional and Doppler TTEs were performed for all patients. All echocardiograms were performed in accordance with the recommendations of the American Society of Echocardiography and were interpreted by level III and/or board-certified echocardiographers. The TTE findings used for the present analysis represent those reported in the final clinical echocardiographic report. The left ventricular (LV) ejection fraction was determined using quantitative methods, either the Teichholz formula from linear dimensions or Simpson’s biplane method of discs. The presence and severity of valvular lesions was determined using an integration of semiquantitative and quantitative methods. Specifically, for the various regurgitant lesions, the quantification of severity included visual estimation, measurement of vena contracta, calculation of effective regurgitant orifice area, and pressure half time measurement, which was left to the discretion of the interpreting physician. For stenotic lesions, the severity was determined by Doppler determination of valvular gradients and calculation of the valve area. The presence of pulmonary hypertension was defined as an estimated right ventricular systolic pressure of 50 mm Hg or greater, which was determined from the maximum tricuspid regurgitation velocity and a standard right atrial pressure of 10 mm Hg. A description of this standard for the definition of pulmonary hypertension has been previously published.


Classification of TTE Findings


We classified abnormal TTEs as those with a “major” abnormality. A major abnormality was defined as the presence of any one of the following: LV ejection fraction less than 45%, LV regional wall motion abnormality, right ventricular dysfunction, pulmonary hypertension (right ventricular systolic pressure of 50 mm Hg or more), moderate or greater valvular regurgitation, significant valve stenosis (aortic stenosis [aortic valve area less than 1.5 cm 2 ], mitral stenosis [mitral valve area less than 1.5 cm 2 ]), moderate or greater pericardial effusion, aortic root (44 mm or greater) or ascending aorta (40 mm or greater) dilatation, or any other significant abnormality (i.e., thrombus, vegetation, tumor, tamponade). A similar classification scheme has been previously published.


In situations in which a previous TTE had been performed, the current TTE report was compared with the previous study. A new significant TTE abnormality was defined as a new abnormality from the preceding list that was not previously noted. A change in the TTE finding was defined as a change of at least one grade in severity from the finding reported from the previous TTE.


Statistical Analysis


The categorical variables for ordering characteristics, patient demographics, and echocardiographic results were compared between each practice site using the χ 2 or Fisher exact test, as required. Continuous variables are reported as the mean ± SD and compared across practice sites using analysis of variance. Statistical significance was indicated by two-tailed P < .05. We calculated that 136 TTEs per inpatient and outpatient group would provide the study with 80% power to detect a 12% between-group difference in the appropriate rate with a two-sided significance level of P < .05. The in-group difference was determined from previously published rates of inpatient (86%) and outpatient (74%) appropriate TTEs using the 2007 AUC. Given the previously published unclassifiable rates of TTE, we assumed 150 TTEs from each practice site would be required for review.




Results


Study Population


We reviewed 476 TTEs from the three practice sites. Of these, 26 TTEs were excluded: 21 because of an inability to access the EMR owing to privacy issues, 3 because the data was insufficient to allow for accurate classification, and 2 because they were ordered for research purposes. Thus, 450 TTEs (150 from each practice site) were included in the present analysis.


Patient Characteristics


The patient characteristics are reported in Table 1 . The TTEs performed in the IAMC were of older patients with a greater number of comorbidities. Specifically, patients in the IAMC had a greater rate of previous myocardial infarction, surgical or percutaneous coronary revascularization, heart failure, and atrial fibrillation.



Table 1

Patient characteristics























































































































































Characteristic Total IAMC OAMC OCHC P value
Patients (n) 450 150 150 150
Mean age (years) 70.6 ± 14.7 73.9 ± 12 72.3 ± 11.1 65 ± 18.7 < .001
Men (%) 50 58 56 37 .004
Medicare (%) 65 77 64 53 < .001
Previous MI (%) 27 35 17 8 < .001
Previous PCI (%) 13 23 13 4 < .001
History of valve surgery (%) 8 9 9 5 .22
History of CABG (%) 11 14 13 7 < .001
Angina (%) 18 29 18 3 .008
Heart failure (%) 28 40 29 16 < .001
Hospitalization within previous 6 months (%) 27 42 27 12 < .001
Diabetes mellitus (%) 21 27 15 21 < .001
Hypertension (%) 78 77 81 74 .31
Hyperlipidemia (%) 68 73 75 57 < .001
Previous or current smoking (%) 44 58 31 44 < .001
Atrial fibrillation (%) 29 39 35 13 < .001
Chronic kidney disease (%) 18 31 14 8 < .001
Cancer (%) 23 33 26 11 < .001
COPD (%) 14 18 9 15 < .001
Peripheral arterial disease (%) 23 29 23 17 .03

MI , Myocardial infarction; PCI , percutaneous coronary intervention; CABG , coronary artery bypass grafting; COPD , chronic obstructive pulmonary disease.


Ordering Provider Characteristics


Cardiologists were the most common ordering provider in the OAMC (66% OAMC vs 37% IAMC vs 11% OCHC, P < .001). Primary care providers most commonly ordered TTEs in the OCHC (86% OCHC vs 46% IAMC vs 21% OAMC, P < .001). Other providers, including surgical specialties, other medical subspecialties (i.e., oncology, pulmonary), and anesthesia, ordered the fewest number of studies at each practice site. The IAMC cohort, OAMC cohort, and OCHC cohort had 90, 75, and 26 unique ordering physicians, respectively.


Echocardiographic Findings


The TTE results from the total study population and stratified by clinical practice setting are listed in Table 2 . Overall, 69% of patients had undergone a previous TTE at any point, and 24% of patients had done so in the previous year. The OAMC had the greatest number of patients with repeat TTEs, followed by the IAMC and OCHC. A major TTE abnormality was found in 49% of the TTEs, with the IAMC having the greatest number of major TTE abnormalities, followed by the OAMC and OCHC. The IAMC was most likely to have TTEs with both a new major abnormality and a change in a known finding.



Table 2

TTE findings






























































































































































Variable Total IAMC OAMC OCHC P value
Patients (n) 450 150 150 150
Mean LVEF (%) 60.6 ± 13.4 56.7 ± 15.6 61.6 ± 13.4 63 ± 11.0 .001
TTE with LVEF <50% (%) 16 25 15 8 < .001
TTE with WMA (%) 20 30 19 11 < .001
Prosthetic valve (%) 5 7 9 5 .29
Aortic stenosis (%) 18 18 23 11 .02
Moderate AS or greater (%) 7 9 13 5 .08
AI (%) 20 17 21 22 .45
Moderate AI or greater (%) 4 2 5 5 .3
MR (%) 41 51 41 31 .002
Moderate MR or greater (%) 12 19 10 8 .01
TR (%) 45 48 49 38 .1
Moderate TR or greater (%) 16 25 13 9 < .001
LVH (%) 27 35 27 19 .01
Congenital (%) 7 2 14 6 < .001
Major TTE abnormality 49 61 50 35 .04
Previous TTE (%) 69 71 83 55 < .001
Previous TTE within previous year (%) 24 36 27 8 < .001
Findings in those with previous TTE
New major TTE finding (%) 36 58 19 32 < .001
Change in finding (%) 42 57 37 29 < .001

LVEF , LV ejection fraction; WMA , wall motion abnormality; AS , aortic stenosis; AI , aortic insufficiency; MR , mitral regurgitation; TR , tricuspid regurgitation; LVH , left ventricular hypertrophy.


In those patients with a previous TTE, 36% had a new major abnormality and 42% had a change in a known finding. Appropriate studies had a greater percentage of new major TTE abnormalities than either uncertain or inappropriate studies (42% vs 24% and 18%, P = .008). Appropriate and uncertain studies had a greater percentage of a change in a known finding than inappropriate studies (44% and 59% vs 26%, P = .005). The percentage of TTEs with new, major abnormalities according to the AUC appropriateness rating and clinical practice site is listed in Table 3 .



Table 3

Major new TTE abnormalities stratified by site and appropriateness rating




























Variable IAMC (%) OAMC (%) OCHC (%) P value
Appropriate TTE 49 19 21 < .001
Inappropriate TTE 30 9 3 .01
Uncertain TTE 33 9 25 .25


TTE Classification


The results of the TTE classification using the 2007 and 2011 AUC are detailed in Table 4 . Significantly more TTEs were classified using the 2011 AUC than the 2007 AUC (98% vs 77%, P < .001). Of the classified studies, the rate of appropriate TTEs was lower using the 2011 AUC than the 2007 AUC (71% vs 83%, P < .001). Additionally, using the 2011 AUC, instead of the 2007 AUC, resulted in a larger number of uncertain studies (7% vs 0%, P < .001), and a nonsignificant trend toward an increase in the number of inappropriate TTEs (22% vs 17%, P = .14).



Table 4

Classification of TTEs using 2007 and 2011 AUC































































































Total (n = 450) 2007 2011 P value
Classifiable (%) 77 98 < .001
Appropriate (%) 83 71 < .001
Inappropriate (%) 17 22 .14
Uncertain (%) 0 7 < .001
IAMC (n = 150)
Classifiable (%) 89 96 .02
Appropriate (%) 89 78 .009
Inappropriate (%) 11 14 .39
Uncertain (%) 0 8 < .001
OAMC (n = 150)
Classifiable (%) 70 99 < .001
Appropriate (%) 70 63 .21
Inappropriate (%) 30 30 .92
Uncertain (%) 0 7 .004
OCHC (n = 150)
Classifiable (%) 73 99 < .001
Appropriate (%) 86 74 .02
Inappropriate (%) 14 21 .14
Uncertain (%) 0 5 .01


Evaluating each clinical site independently, the 2011 AUC classified a greater number of TTEs at each location ( Table 4 ). Of the classified studies, the rate of appropriate TTEs at the IAMC and OCHC was significantly lower using the 2011 AUC than the 2007 AUC. In all practice settings, the rate of uncertain studies was significantly greater when the 2011 AUC were used.


The details of the classification and reclassification of TTEs by practice site using the 2011 AUC are listed in Table 5 . Compared with the 2007 AUC, application of the 2011 AUC resulted in reclassification of 27% of all TTEs. Across all practice settings, most reclassified studies went from unclassifiable with the 2007 AUC to classifiable with the 2011 AUC. Overall, of the newly classified studies, 43% were classified as inappropriate and 19% as uncertain, with the remainder classified as appropriate. The OAMC had the most number of studies change from unclassifiable to classifiable using the 2011 AUC. Only a small percentage of studies classified as appropriate or inappropriate with the 2007 AUC were reclassified to a different appropriateness rating.



Table 5

TTE classification and reclassification by practice site


































































































































Variable Total IAMC OAMC OCHC P value
Patients (n) 450 150 150 150
Classifiable by 2007 AUC (%) 77 89 70 73 < .001
Appropriate (of those classifiable) (%) 83 89 70 86 < .001
Inappropriate (of those classifiable) (%) 17 11 30 14 < .001
Uncertain (of those classifiable) (%) 0 0 0 0
Classifiable by 2011 AUC (%) 98 96 99 99 .09
Appropriate of those classifiable (%) 71 78 63 74 .01
Inappropriate of those classifiable (%) 22 14 30 21 .004
Uncertain of those classifiable (%) 7 8 7 5 .59
Change between 2007 and 2011 AUC (%) 27 15 35 31 < .001
Unclassifiable to classifiable (%) 21 7 29 27 < .001
Unclassifiable to appropriate (%) 9 3 12 12 .004
Unclassifiable to inappropriate (%) 9 3 13 10 .005
Unclassifiable to uncertain (%) 4 2 4 5 .43
Appropriate to inappropriate (%) 2 2 1 3 .71
Inappropriate to appropriate (%) 1 0 2 1 .72
Inappropriate or appropriate to uncertain (%) 3 6 3 1 < .001

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Jun 2, 2018 | Posted by in CARDIOLOGY | Comments Off on Comparison of the 2007 and 2011 Appropriate Use Criteria for Transthoracic Echocardiography in Various Clinical Settings

Full access? Get Clinical Tree

Get Clinical Tree app for offline access