Background
Cardiac sonographer credentialing may guarantee baseline content knowledge but does not directly evaluate clinical scanning skills. The aim of this study was to determine the relationship of sonographer credentialing status to clinical competence, as defined by image quality of case studies submitted for Intersocietal Accreditation Commission (IAC) adult transthoracic echocardiography laboratory accreditation.
Methods
In this study, data were retrieved retrospectively from the IAC Echocardiography application database for all adult echocardiography laboratories applying online from August 10, 2011, to December 12, 2013. Aortic stenosis (AS) and left ventricular (LV) regional dysfunction case studies were analyzed separately. Sonographers submitting case studies were coded as credentialed or noncredentialed. An image quality score (IQS) was calculated for each case study, reflecting review scores for examination components directly related to image quality and acquisition. The group of sonographers was divided into quartiles on the basis of annual procedure volume; mean case study IQS was compared between credentialed and noncredentialed sonographers.
Results
For all four quartiles of the LV cases and the lowest three volume quartiles of the AS cases, mean IQS was significantly higher for credentialed than noncredentialed sonographers. Mean IQS in the highest volume quartile for AS studies was not significantly different by credential status.
Conclusions
In the setting of IAC Echocardiography accreditation, credentialed sonographers achieved higher mean IQSs than noncredentialed sonographers in seven of eight comparisons. However, further research will be required to expand the scope of this inference beyond AS and LV regional dysfunction cases submitted for IAC adult transthoracic echocardiography laboratory accreditation.
Highlights
- •
In the setting of IAC Echocardiography accreditation, credentialed sonographers achieved higher mean IQSs than noncredentialed sonographers in seven of eight comparisons.
- •
The findings of this study may support the long-standing public and professional assumption that sonographer credentialing ensures clinical competency.
- •
Further research will be required to expand the results of this study beyond AS and LV regional dysfunction cases submitted for IAC adult transthoracic echocardiography laboratory accreditation.
The American Society of Echocardiography (ASE) views a professional sonographer as a “highly skilled and well-educated person.” This broad definition encompasses clinical competence (scanning skills), content knowledge gained through education and experience, and attainment of a sonography credential. Indeed, the ASE and the Society of Diagnostic Medical Sonography strongly recommend that all sonographers entering the field obtain nationally recognized credentials. The ASE and the Society of Diagnostic Medical Sonography not only appear to view acquisition of a cardiac sonography credential as a guarantee of baseline content knowledge but also seem to imply that possession of a credential ensures clinical competency. In addition, Medicare and third-party payers increasingly tie echocardiographic examination reimbursement to sonographer credentials, laboratory accreditation, or both. However, sonographer credentialing examinations do not directly evaluate clinical scanning skills. Instead, prerequisites to qualify for examination appear intended to ensure that applicants possess baseline levels of clinical competence.
To achieve Intersocietal Accreditation Commission (IAC) Echocardiography laboratory accreditation, a facility must demonstrate that sonographers possess minimum levels of training and experience. Laboratories are also required to establish processes by which all sonographers will eventually become credentialed. However, current qualification pathways allow noncredentialed sonographers (with the exception of the technical director) to perform echocardiographic examinations in accredited laboratories.
IAC Echocardiography accreditation also evaluates cardiac sonographer clinical competency through the submission of representative echocardiography case studies. According to ASE, basic skills required for clinical competence in echocardiography include knowledge of ultrasound physics and instrument settings, hemodynamics, and normal and abnormal cardiac anatomy and physiology. Sonographers must appropriately assess patients’ clinical histories, evaluate normal and abnormal findings, tailor echocardiographic examinations as required, and communicate pertinent results to interpreting physicians. For most studies, the cardiac sonographer bears sole responsibility for the technical quality and diagnostic content of the examination.
Adult laboratory accreditation requires submission of transthoracic echocardiographic (TTE) cases representing two pathologic processes: (1) left ventricular (LV) regional dysfunction as demonstrated by regional wall motion abnormalities due to coronary artery disease or myocardial infarction and (2) native valvular aortic stenosis (AS). Assessment of LV function and regional wall motion is an essential component of all echocardiographic examinations. Assessment of AS requires additional complex evaluation of valvular structure, function, and hemodynamics. Indeed, Wiegers noted that accurate assessment of the severity of AS is technically demanding. Review and scoring of the cases submitted for accreditation allows evaluation of the technical quality of a facility’s echocardiographic examinations. Subsequently, the case review, as an assessment of technical skill, is heavily weighted in the accreditation decision.
Currently, the assumption that sonographer credentialing ensures clinical competency has received little, if any, attention in the literature. IAC accreditation provides an avenue by which this assumption may be tested, because specific clinical skills are evaluated for both credentialed and noncredentialed sonographers. The purpose of this study was to determine the relationship between IAC Echocardiography accreditation case study image quality score (IQS), which may be considered a measure of clinical competence, and credentialing status of the sonographers performing those cases. The study hypothesis was that credentialed sonographers have higher mean IAC Echocardiography accreditation case study IQSs than do noncredentialed sonographers for evaluation of both LV regional dysfunction (wall motion abnormalities) and AS.
Methods
In this correlational study, data were retrieved retrospectively from the IAC Echocardiography application database. The IAC Echocardiography application database contains all data entered into the online application, as well as the ratings assigned by application reviewers for case submission evaluation. The secure IAC Echocardiography online application requires the completion of questions that gather laboratory demographic and operations data and allows reviewers to evaluate compliance with IAC Echocardiography standards. During accreditation, case submissions, personnel qualifications and supervision, facility safety and resources, equipment, reports and records, administrative components, diagnostic testing modalities, and quality improvement processes are evaluated and rated.
Content validity of the application database has been established through development of accreditation standards on the basis of published professional guidelines, peer-reviewed literature, and consensus by the Board of Directors, which is composed of experts from sponsoring organizations. There are no similar instruments available for concurrent or predictive assessment of database validity. However, Nagueh et al . recently used the IAC database to identify common deficiencies and explore predictors of delayed accreditation with regard to key quality measures defined by IAC Echocardiography and the professional echocardiography community. The key variable of image quality included the same components as the IQS, and predictors of accreditation delays included missing required views and incomplete Doppler evaluation of AS.
To evaluate database reliability, a reviewer reliability study was conducted in 2013 (W. Judd, personal communication, September 16, 2013). A representative adult TTE accreditation application containing four adult TTE case studies was created by the director of accreditation in the Echocardiography Division and was used as the control application. Three full-time IAC Echocardiography staff members scored the application independently and then met to compare ratings and resolve any differences. The outcome of this process was a single rating (the control rating), which became the foundation for establishing reviewer reliability. The control application materials and case studies were reviewed by 35 IAC Adult Echocardiography reviewers and scored using an online portal.
Reviewers’ agreement was calculated by evaluating the percentage agreement of reviewer ratings with the control rating (W. Judd, personal communication, September 16, 2013). Most of the reviewers had between 85% and 90% agreement with the control rating. Overall, the rating accuracy was 92%, demonstrating very good consistency among reviewers. In addition, analysis of item agreement was performed; most items showed >80% agreement with the control rating, which supported use of individual imaging case review questions for this study.
Research approval for the present study was obtained from the Saint Mary’s University of Minnesota Research Review Board and the IAC research director. Requested data were retrieved from the IAC Echocardiography application database in Ellicott City, Maryland, by IAC research staff members. Data were anonymized and summarized in Microsoft Excel (Microsoft Corporation, Redmond, WA) and SPSS (IBM, Armonk, NY) files.
All cases submitted from echocardiography laboratories applying online for IAC Echocardiography adult TTE accreditation from August 10, 2011, to December 12, 2013, were included. Only studies from laboratories with protocols meeting the required standards were included, because specific images missing from case studies may be related to an incomplete protocol rather than sonographer skill. Excluded from this group were cases from laboratories without sonographers ( n = 15), cases that could not be evaluated by the case reviewers for technical reasons ( n = 62), and cases in which the self-reported sonographer annual procedure volume exceeded a plausible upper limit of 5,000 studies per year ( n = 21). Cases submitted incorrectly that did not demonstrate LV regional dysfunction (wall motion abnormalities) or AS ( n = 669) were also excluded, because these cases are not evaluated by case reviewers.
Data on sonographer credentialing status and case study image quality, as scored by trained application reviewers, were extracted from the database for analysis. Although sonographer experience doubtless has a role in clinical competence, data identifying the length and type of sonographer work experience were not available from the database. Indeed, simply documenting years of employment may not accurately reflect sonographer experience because of variations in full-time versus part-time status or ancillary performance of noncardiac ultrasound examinations. Therefore, average annual echocardiographic procedure volumes documented in the database for each sonographer were used as an indicator of work experience.
The IQS was the summary score for components evaluated during each case review that were directly related to image quality and acquisition ( Box 1 ). IQS components evaluated during the case review represent key images identified by IAC Echocardiography that allow integration of two-dimensional, color, and Doppler data for comprehensive evaluation of cardiac structure and function. Well-aligned two-dimensional images should be viewed in standard tomographic planes and provide adequate endocardial definition to accurately assess cardiac structure and function. Acquisition of Doppler data should include appropriate alignment of the ultrasound beam to reduce angle error. In patients with valvular regurgitation, a minimum of two imaging planes should be used to evaluate the severity of regurgitation. For AS, the acquisition of the highest flow velocity across the stenotic valve should include interrogation from at least three echocardiographic windows using the nonimaging continuous-wave Doppler transducer. Doppler estimation of right ventricular systolic pressure should always be attempted in multiple views when tricuspid regurgitation is present but is particularly important in patients with pulmonary hypertension.
| Image quality components for LV and AS cases |
| Presence of right ventricular inflow views |
| Presence of required parasternal short-axis views |
| No apical view foreshortening |
| Color Doppler evaluation of the mitral valve in at least two views |
| Endocardium clearly defined in 14 of 16 segments |
| LV outflow tract pulsed Doppler sample volume placed in the correct location |
| Doppler velocity of tricuspid regurgitation (when present) is recorded in at least two views |
| Image quality components for AS cases only |
| Peak aortic valve Doppler velocity acquired using the dedicated continuous-wave transducer in at least two standard views |
| Clear envelope obtained from a nonapical window using the dedicated continuous-wave Doppler transducer |
| Suprasternal notch views demonstrated according to the facility protocol |
Each image quality component was scored 1 if it met the case review standard and 0 if it did not. If tricuspid regurgitation was not present, the case review standard for the tricuspid regurgitation component was considered met, and the null value in the data set was scored 1. LV regional dysfunction cases were scored from 0 to 7; AS cases were scored from 0 to 10.
Case studies were categorized as either LV regional dysfunction (wall motion abnormalities) or AS, which were analyzed separately. Every sonographer submitting a case study was coded as credentialed or noncredentialed. Subsequently, the mean case study IQS for each pathology group was calculated for the credentialed and noncredentialed sonographer groups, with division into quartiles on the basis of annual procedure volume.
Descriptive data for this study included the number of laboratories applying for accreditation, the number of sonographers per laboratory, and laboratory accreditation status. Levene’s Test for Equality of Variances was performed to ensure that the credentialed and noncredentialed sonographer groups were homogeneous regarding the distribution of average annual procedure volumes (an indicator of work experience) around the group mean. Because the groups were not homogeneous, the LV regional dysfunction and AS case groups were divided into quartiles, and each quartile was evaluated for homogeneity of annual procedure volumes. Subsequently, parametric two-sample t tests were used to evaluate scoring differences (mean ± SD IQS) between the credentialed and noncredentialed sonographer groups. A one-tailed P value < .05 was accepted as indicating statistical significance. All analyses were performed using SPSS Statistics 22.
Results
Demographic Data
A total of 2,369 laboratories were represented; the number of sonographers per laboratory ranged from one to 83, with a mean of 3.6 and median of two. Accreditation was granted to 38.9% of applicant laboratories ( n = 921). Data analysis included 15,264 adult TTE case studies performed by 7,215 sonographers; noncredentialed sonographers performed 14.5% of those studies ( n = 2,217). A total of 7,299 cases were classified as LV regional dysfunction and 7,965 as AS.
Sonographer Characteristics
Levene’s Test for Equality of Variances showed a highly significant difference between the credentialed and noncredentialed sonographer groups ( t = 7.167, P < .001). Consequently, both groups were divided into quartiles for further analysis on the basis of the number of cases performed per year, as follows: quartile 1, one to 540 cases; quartile 2, 541 to 900 cases; quartile 3, 901 to 1,250 cases; and quartile 4, 1,251 to 5,000 cases.
LV Regional Dysfunction Cases
Of the 7,299 LV regional dysfunction cases, 1,123 (15.4%) were performed by noncredentialed sonographers, with homogeneity of annual procedure volumes demonstrated for each quartile. Subsequently, testing for independent means showed that credentialed sonographers had a significantly higher mean IQS (all P ≤ .01) in each quartile ( Table 1 ).
