Reports that cardiologists overuse procedures are common in the lay press, with concerns ranging from legitimate differences in judgment to outright fraud. Clinical practice guidelines were created to assist physicians in practicing evidence-based medicine, but in 2004, on the basis of mounting evidence of unexplained geographic variation in use and increasing cost and regulatory pressures, the American College of Cardiology decided to create a new category of standards, appropriate use criteria (AUC). AUC are constructed around common patient scenarios and intended to inform bedside decision making and provide guidance regarding overall patterns of care. AUC blend all levels of evidence gleaned from relevant randomized clinical trials, clinical practice guidelines, and expert consensus. AUC are based on levels of evidence A (randomized controlled trials) and B (observational studies) when available (and including level of evidence C, expert opinion). The evidence is considered following a rigorous, prospective process adapted from an existing Delphi methodology used by the RAND Corporation for similar purposes, which generates ratings regarding the reasonableness of performing a test or procedure in a specific clinical situation. Importantly, the technical panel rating the indications in terms of appropriateness is composed of <50% of specialists in that area.

The reactions to these early documents ranged from embracing them as a useful and needed tool to guide practice to outright rejection (“How dare anyone imply that I practice inappropriate medicine!”). The first applications of AUC to actual patients provided interesting and important insights into current patterns of care, highlighted areas of possible overuse, and indicated that quality improvement was possible by means of targeted intervention, especially because most inappropriate use was clustered into just a few indications. However, there were also questions: the complexities of clinical medicine meant that the AUC class often could not be determined, and when it could, reproducibility among observers was modest. There were subtle differences in indications and rankings between similar modalities such as stress echocardiography and stress nuclear medicine that could be confusing. Those wishing to adopt AUC were uncertain as to the “correct” level of inappropriateness and what the optimal relationship should be between AUC ranking and the likelihood of a new finding or a change in care. Clearly, this was an effort in its infancy.

With time, experience, and a broader awareness of the need for such documents, AUC have matured. Revisions of virtually all the imaging AUC have increased the documents’ coverage of the full range of clinical scenarios, harmonized indications and methods, and in the case of echocardiography created a single combined document for easier reference. By including an updated evidence base and addressing omissions in prior documents, the revised echocardiographic AUC nearly doubled the number of indications from 110 to 202, with 97 indications rated as appropriate (up from 66, a 47% increase), 34 as uncertain (from 11, a 209% increase), and 71 as inappropriate (from 33, a 115% increase). In addition, the revised document developed a guiding set of principles that broadly define an approach to testing on the basis of five general types of indications: those encompassing initial diagnosis, guidance of therapy, or evaluation of a change in clinical status were generally ranked as appropriate, while early follow-up was more likely to receive uncertain or inappropriate ratings than late follow-up, especially in asymptomatic patients (i.e., surveillance).

A second wave of studies examining the applicability of the AUC to clinical echocardiography is now being published; this issue of JASE provides three such examples, which together demonstrate significant progress as well as a continued need for refinement. Both studies by Bhatia et al. and that by Mansour et al. compare the application of the older AUC with the new revision and demonstrate substantial improvement in the ability to classify the real-world uses of all types of echocardiography (transthoracic, transesophageal, and stress) into AUC rankings. Indeed, in these reports, 97% to 100% of all indications can now be mapped to the revised AUC, depending on modality, with robust reproducibility. The remaining “missing” transthoracic echocardiographic indications identified by the present studies are largely limited to those encountered in high-end referral centers (transplantation and adult congenital heart disease) and infrequently seen by the vast majority of echocardiographers. The gaps also reflect ongoing innovations in care, such as transcatheter interventions for structural heart diseases, for which there are few data or little consensus on optimal imaging use. For transesophageal echocardiography, areas such as perioperative echocardiography are outside the scope of the present documents and may be specialized enough to merit stand-alone AUC. Thus, aided by the identification of gaps when applying the older AUC to clinical practice by these and other authors, a major concern has been addressed, which has made the AUC much more robust and enhanced their utility and relevance. Similarly, future revisions will benefit from the gaps identified in the present documents.

Interestingly, all three reports also note decreases in the percentage of studies that are ranked as appropriate across modalities. Although initially counterintuitive (has our decision making really gotten worse?), all three reports are consistent in noting that nearly two thirds of echocardiographic studies that are newly classifiable by the revised AUC are ranked as uncertain or inappropriate. This is not surprising, as the update added nearly twice as many indications in these categories than were added to the appropriate category. Because nearly all echocardiographic studies (96%) classifiable in both sets of AUC mapped to the same category of appropriateness in both, both sets of authors suggest that the earlier documents may have underestimated the prevalence of the uncertain and inappropriate categories in clinical practice and overestimated the prevalence of the appropriate category.

One of the most important contributions of AUC is the identification of the common inappropriate indications, because this in turn provides the basis for education and quality improvement efforts. In applying the 2011 criteria, Bhatia et al. (for transthoracic echocardiography) found that the most common inappropriate uses for outpatient transthoracic echocardiography centered on the routine reevaluation of chronic conditions in the absence of a change in status, including mild valvular stenosis, hypertension, left ventricular function in those with prior results showing normal function, ascending aortic dilation, and completely repaired congenital heart disease in adults. For inpatients, suspected endocarditis, pulmonary embolism, and syncope in patients clinically at low risk were also common inappropriate indications. For transesophageal echocardiography, they found the most common inappropriate indications to be diagnosis of endocarditis in patients with low pretest likelihood, use of transesophageal echocardiography instead of transthoracic echocardiography that was likely to provide adequate data, or when no change in treatment was anticipated regardless of the findings. Unfortunately Mansour et al. do not provide information on the most commonly used inappropriate indications in their report, as their recognition can provide powerful guidance to ordering providers regarding situations in which echocardiography may not be useful in the majority of patients and in designing focused interventions to improve test ordering and sharpen our appreciation of the proper uses of imaging. However, this must be done with caution. As the AUC documents themselves note, these are intended as only as guides rather than rules, and AUC are not intended to be used to deny needed imaging care in individual patients. Conversely, a rating as appropriate should not be misunderstood as a recommendation to image every patient who might fit that scenario but rather that it would be reasonable to do so if the information derived will assist in care.

Although there is general agreement that providers should be knowledgeable about their ordering patterns and should minimize the use of imaging for inappropriate indications, there is no evidence as yet as to whether that would improve processes of care, costs, and/or outcomes. This is a complex question, especially for a diagnostic test, because test performance alone cannot directly affect outcomes; moreover, the science of outcomes research in imaging is still a matter of debate. However, this question can be explored by examining the likelihood of new or major findings for each modality according to the AUC ranking, as Mansour et al. and Bhatia et al. have done. By far the highest likelihood of a new or major finding was in echocardiographic studies performed for appropriate indications, with a lower likelihood in studies performed for uncertain indications and the lowest likelihood for studies done for inappropriate indications. Although this gradient is reassuring, the correct and much harder evidentiary standard to achieve is documentation that an echocardiographic exam improves outcomes and does not merely confirm the presence of known major findings. Even the discovery of new findings is at best an intermediate measure; to be of value, echocardiographic findings should directly influence care in some way. Also, we must be careful not to equate new findings with imaging value, for this ignores the importance of negative results, which may exclude suspected disease or reassure patients and their physicians that disease is not progressing and that current care is adequate. Indeed, Mansour et al. note that more than twice as many transesophageal echocardiographic studies influenced medical care than had new findings. Although quite complex, defining the value of imaging is a critical question, especially in this era of constrained resources. AUC may provide essential standards to guide future research in this area, and subsequent revisions would certainly benefit from such studies.

Both Bhatia et al. and Mansour et al. document new or major findings in a handful of inappropriate studies. These “positive” studies appear to question the validity of an inappropriate rating; however, perhaps the answer lies in the knowledge that any imaging study may yield an incidental finding but that their incidence and significance are generally low. As with all incidental findings, knowledge of their presence may not improve care but instead lead to further evaluation, which may be unnecessary. Indeed, the constancy of the relative incidences of major findings in each of the AUC categories in the original and revised AUC lends validity to this hypothesis as well as to the categories themselves.

Another concern regarding studies based on ranking echocardiographic studies that have already been or will be performed is that it cannot address the appropriateness of those studies that were not performed, whether through underuse of needed imaging, consultation resulting in a revised diagnostic plan as may be commonly the case with transesophageal echocardiography, or administrative denial or some other mechanism. This in turn highlights an important aspect of care, underuse, that is often not well addressed by AUC. This is a very difficult question to address, because the indirect link between a diagnostic test and outcomes is also relevant in this situation, when important findings are missed and not available to guide care when a needed test is not performed. Thus, it is difficult to trace the connection between a missed test and a poor outcome. Nevertheless, underuse clearly exists, as shown by two recent studies. Ballo et al. applied the revised 2011 echocardiographic AUC to hospitalized patients who were discharged without having undergone transthoracic echocardiography and concluded that in 16% of them, it would have been appropriate to have ordered transthoracic echocardiographic evaluations. In another study, Curtis et al. demonstrated that more than one third of patients with newly diagnosed heart failure failed to receive imaging tests that could quantify ejection fraction, which is one of the handful of American College of Cardiology and American Heart Association performance measures identifying “must-do” care. Furthermore, the proper use of testing was associated with decreased mortality. However desirable it might be to do so to ensure optimal imaging care, AUC, as currently designed, are unable to address the “appropriateness” of not performing echocardiography.

Additional important concerns are raised by past and current applications of AUC: what is the “correct” level of appropriate (or inappropriate) use, and how much variation is acceptable? This is a critical issue, because reimbursement is increasingly tied to demonstrating quality of care, and in the absence of more robust outcomes data from imaging, process-of-care measures such as AUC are likely to become the default yardstick. Bhatia et al. ’s comparison (for transthoracic echocardiography) of academic and community practices demonstrates that patient mix may affect a provider’s AUC “performance”: a higher rate of inappropriate use was found in an academic medical center compared with a community practice. This was felt to reflect differences in patient populations, with the academic practice having a higher proportion of patients with established cardiovascular diagnoses who were more likely to have been tested previously, but one could argue that some of these repeat studies may have been unnecessary. Furthermore, side-by-side comparison of the reports in this issue of JASE reveals differing levels of appropriate use even at two academic medical centers, a difference found in applying both the original and revised AUC. The data suggest that physicians in Boston may be more inappropriate than those in Chicago (22% of transthoracic echocardiographic studies performed in Boston were inappropriate vs 11% performed in Chicago; for transesophageal echocardiography, these percentages were 8% and 3%). Certainly this is of a piece with the notoriously rowdy Red Sox Nation, but perhaps enthusiastic Cubs fans might take exception.

Together, Bhatia et al. ’s and Mansour et al. ’s reports demonstrate substantial improvement in the echocardiographic AUC, reflecting maturation consistent with the growing acceptance and adoption of AUC. Nevertheless, the future still holds many questions and challenges. There is no doubt that the methods, nomenclature, indications, and application of AUC will continue to be refined. Continued research on the optimal use of imaging will also inform this field, although given the difficulties in randomized trials in this area, the evidence base will likely grow largely because of observational studies and registries. These efforts need to be prioritized and funded. New research findings will enable upgrading the level of evidence as more robust data are generated and will serve to resolve the rankings of some indications that are currently classified as uncertain. Most important, research is critically needed to validate indication rankings and demonstrate that use of AUC really does improve outcomes.

We also urgently need guidance as to how to best use AUC in clinical care. Implementation research can evaluate potential AUC quality metrics and how they may be used to evaluate and guide care. Designed as tools to track overall patterns of care and to identify areas for education and improvement, AUC have perhaps been most frequently applied in case-by-case decision-making, whether by physicians or payers. Certainly the need for guidance at the bedside is real, and the extent to which AUC can contribute to this is of great value. Their utility for this purpose would be enhanced by the use of decision support tools and other positive interventions, rather than by care denials. Development and implementation testing of such tools is under way, and it is critical that this continue.

The current push to value-based purchasing and away from the fee-for-service model opens up another area in which the application of AUC may be of substantial value, but evidence is lacking. As care pathways and episodes of care are designed to enable bundled payments, echocardiography will be seen as a cost center and no longer a potential source of revenue. AUC rankings can provide needed documentation of the value of echocardiography and could help preserve access for patients who would benefit from testing.

In conclusion, the three studies in this issue of JASE together demonstrate that the revision of the echocardiographic AUC show much progress over the original documents. AUC have become an important set of clinical and policy standards at a time when these are sorely needed. However, they have not achieved perfection. Many, if not all, of these shortcomings are being addressed by our clinical, research, and policy communities, which, together with patients, are all stakeholders who will benefit from the improvements in AUC.