The value of echocardiography for diagnosis, therapy, risk stratification, clinical investigation, and basic research is well established. However, as we enter a period of increased scrutiny on resource utilization, the challenge now is to understand how best to allocate resources for cardiac imaging and when to perform echocardiography. Thus, attention has turned to the delivery of “quality” care. Quality in cardiovascular imaging has been the subject of summits and “think tanks” designed to define components of quality.

The American Society of Echocardiography has championed efforts to delineate the multiple facets of quality echocardiography laboratory operations. Numerous factors constitute quality in echocardiography, including the laboratory structure and the imaging process. The laboratory structure includes the physical space and equipment, as well as the personnel (sonographers and physicians). The imaging process can be further separated into five main areas that may influence patient outcomes: patient selection, image acquisition, image interpretation, results communication, and the incorporation of results into care. Most agree that measuring and improving the quality of echocardiography labs is important as we aim to provide the best service possible. But how we should measure quality, and whether quality service makes a difference for outcomes in patients, are important and largely unanswered questions.

There are many approaches to measuring quality in health care. For example, the patient experience can be examined to determine if patients are satisfied with the care they receive. Alternatively, process-of-care measures can be evaluated to determine if providers are following practice guidelines that have been identified as recommended or appropriate care. Furthermore, one can examine risk-adjusted outcomes to determine how patients are faring. Even outcomes such as reduced downstream testing, improved correct diagnosis rate, and improved health status can be quantified and compared.

Accreditation of laboratories, certification of echocardiography physicians, and credentialing of sonographers are all important and assumed to be markers of quality; however, to date, there has been little information on whether these are truly associated with quality care. In fact, research examining the impact of such benchmarks on patient outcomes is limited. It is well documented that echocardiographically derived parameters of cardiac structure and function, such as left ventricular (LV) ejection fraction (LVEF) and the degree of mitral regurgitation, have important prognostic significance. Furthermore, many treatment algorithms for device-based or procedure-based intervention in cardiovascular disease are determined by echocardiographically derived parameters, such as LVEF. However, whether recommended benchmarks of laboratory and interpreting physician quality are associated with the accuracy of echocardiography interpretation is unknown.

In this issue of JASE , Heidenreich et al. , supported by the American Society of Echocardiography’s Outcomes Grant Program, report a novel method for measuring quality in echocardiography. Specifically, in 63,108 patients who had undergone transthoracic echocardiographic (TTE) imaging at one of four academic medical centers, they examined the relationship between LVEF and all-cause mortality. The premise was that because there is a clear relationship between LVEF and mortality, this relationship can be examined for the echocardiograms interpreted by each physician, and the influence of various factors (such as training, experience, and certification) on this relationship can be determined. Thus, those interpreting physicians whose LVEF-versus-mortality relationships are stronger would be identified as higher quality interpreting physicians. Stated differently, mortality was used as a marker of the accuracy of LVEF measurement. Forty interpreting physicians and 56 sonographers were included in the analysis. The impact of physician and sonographer characteristics on LVEF predictive accuracy was determined by comparing the areas under the receiver operating characteristic curves for prediction of 1-year mortality with LVEF. The investigators found that physicians who were board certified by the National Board of Echocardiography (NBE) had a significantly higher mean area under the curve compared with those not certified. The interaction of LVEF and board certification remained significant after adjustment for patient characteristics. In addition, the area under the curve was greater for final physician-reported LVEF (visual estimate informed by the method of disks) compared with the LVEF that could be calculated solely from the sonographer-measured LV dimensions using the Teichholz formula. It is important to point out that the relationship between LVEF and 1-year mortality was present for all echocardiographic readers; however, this relationship was strongest for NBE-certified physicians. Another key feature of the results is that the NBE-certified physicians were more accurate in distinguishing normal from mildly abnormal LVEFs. An important message is that most echocardiographers do well at identifying normal LVEFs and significantly impaired LVEFs, but high-quality readers can also discriminate the subtle abnormalities in LV systolic function. There were no differences between physicians on the basis of years since training, level of training (level I, II, or III), or number of TTE studies reviewed per year.

Perhaps the greatest strength of the study by Heidenreich et al. is the novel attempt to define and measure quality. Previous investigations have evaluated how noninvasive cardiac imaging studies influence patient management, and data indicate that further testing (i.e., cardiac catheterization) and medication changes increase in proportion to the degree of abnormality on noninvasive imaging studies. Additionally, studies of the appropriateness of echocardiography document that new, important abnormalities on TTE imaging are more likely to be present on echocardiographic studies classified as appropriate, compared with those considered inappropriate. The question that Heidenreich et al. address is a more proximal issue than that addressed by other studies of cardiac imaging and outcomes. Namely, is the relationship of an abnormal finding and patient outcome mediated by the quality of the interpretation of the imaging study? A unique aspect of this approach is that mortality is an objective measure and an improvement over analyses based on comparison with an expert’s assessment. Also, the approach used in this study did not require that multiple or duplicate tests be performed on the same patient.

The finding that NBE-certified echocardiographers’ LVEF determinations were associated with a stronger relationship with mortality indicates that NBE certification may represent a useful quality metric for echocardiography laboratories. The NBE examination was designed with the primary objective of providing practicing echocardiographers the opportunity to demonstrate special competence in echocardiography on the basis of an objective standard. The results of this study are an external validation of the NBE examination and certification process.

Because one approach proposed by payers is to reimburse providers only for those echocardiograms that meet quality standards, questions have arisen as to what benchmarks identify quality. The methodology in this study could be used as one component in the search to identify characteristics of quality echocardiography. The determination of quality in imaging, however, is a complex issue. Although it is easier to assess adherence to guidelines and appropriate use criteria, analyzing the impact on clinical outcomes, as investigated by Heidenreich et al. , is a critical step.

As the authors acknowledge, there are issues that cannot be addressed from this data set. First, although a large number of TTE studies (>63,000) were included in the analysis, the numbers of physicians ( n = 40) and sonographers ( n = 56) were relatively small. Furthermore, the majority of physicians (85%) were level II trained, with only a very small number trained at levels I and III. Therefore, the ability to ascertain differences among the levels of training in physician interpreters of echocardiograms was limited. In fact, one could conclude that the results of this study pertain only to level II–trained physicians. Because only four laboratories were involved in this study, the role of lab accreditation could not be assessed. Also lacking is the ability to assess the role of the combination of level of training and NBE certification (i.e., level II training with NBE certification vs level III training alone). Similarly, only approximately 20% of the sonographers had <5 years of experience, which may have limited the ability to detect differences on the basis of years of practice for sonographers (although a trend toward those with more experience having greater accuracy was found). Finally, the study included TTE studies from four academic centers (two US Department of Veterans Affairs hospitals and two nongovernment centers), so the ability to generalize their findings to nonacademic settings may be limited.

In addition, in this study, one is unable to assess how much of the accuracy of the LVEF determinations is due to image quality. We cannot determine from the study if the NBE-certified physicians were better at reading the more technically limited images or if their LVEF determinations were more accurate across the entire range of image quality. It may also be desirable to use this methodology on a larger data set composed of academic and community echocardiography labs to determine the influence of various combinations of markers of quality (e.g., comparing a level I–trained physician in a laboratory accredited by the Intersocietal Accreditation Commission with a level II–trained physician in an unaccredited lab). In fact, one can envision that with a large enough sample of echocardiograms, physicians, and labs, one might be able to identify the combination of quality markers that provides the best area under the receiver operating characteristic curve.

The method of LVEF estimation used in this study was described as a visual estimate that was informed by a calculation on the basis of method of disks. This is in contrast to the American Society of Echocardiography’s recommendation, which favors quantitation that then may be tailored by the visual estimate. This subtle difference that overweights the subjective assessment of LV function might explain why the certified physicians were better at the borderline LVEF determinations.

Because accurate determination of LV systolic function is a key component of a quality echocardiographic study, the follow-up question, then, is how practitioners can improve their ability to assess LVEF and thus become higher quality echocardiographic readers. Formalized teaching interventions have been designed that significantly reduce the interobserver variability in visual estimation of LVEF by sonographers and physicians. Similar programs have been developed for categorizing aortic regurgitation and the assessment of diastolic function.

In summary, there are many reasons why the current study by Heidenreich et al. is important. A simple one is that as patients become more responsible for their care, they will look for benchmarks of quality, such as accreditation, certification, and credentialing. Such information is easy for patients to obtain on the internet and from other sources. In addition, as mentioned above, such metrics will become easy markers for payers to use. The results of this study may provide some justification for these approaches, although further study in larger, more diverse populations of physicians and sonographers is necessary. Additionally, one can envision that this methodology could be applied to develop quality metrics for echocardiographic parameters beyond LVEF. For example, a similar scheme could be devised in the context of valvular heart disease, such as the impact of mitral regurgitation severity grade on mortality or the impact of the calculated aortic valve area on subsequent aortic valve replacement.

Despite the growing evidence base for quality in echocardiography and noninvasive cardiac imaging in general, future challenges remain. We must demonstrate that quality practice does not cost more and perhaps has beneficial effects on downstream costs. Additionally, although the current study adds to the manner in which we may define quality, a quality echocardiography laboratory relies on many factors, and it is unlikely that a single metric will be sufficient to represent “quality.” Rather, a multifactorial approach, similar to what has been used in other forms of clinical medicine, will be necessary to assess the quality of imaging practices. Ensuring that qualified physicians interpret echocardiograms is just one of many facets of the imaging process for which we need to apply quality measures and demonstrate that they matter. Heidenreich et al. have taken a first step in quantifying how image interpretation can affect quality.