Baseline and Postintervention Performance Sets

Using laboratory logbooks, studies for analysis were selected from all the complete transthoracic echocardiographic studies (Current Procedural Terminology codes 93306 and 93307) performed at SHVI during a randomly chosen week for each phase of the study between October 2008 and December 2009. During the study period, approximately 42,000 studies were performed and interpreted at SHVI. Study sets were taken at baseline, after the educational intervention (post–educational intervention set), and in a final set to determine outcomes from interventions (final outcomes set). For the baseline set, the first chronologic study for each interpreting physician was chosen. After ensuring the representation of all interpreting physicians, sonographers not yet included in the sample had their first chronologic studies chosen for inclusion. For the two postintervention sets, studies were chosen in a similar fashion, with at least three studies for each sonographer and interpreting physician. All sonographers and interpreting physicians were included in the study. Both inpatient and outpatient studies were used. Studies were performed on either GE Vivid 7 machines (GE Healthcare, Milwaukee, WI) or Philips iE33 machines (Philips Medical Systems, Andover, MA) with 3.5-MHz transducers. A total of 50 studies constituted the baseline set, with 110 in the post–educational intervention set and 109 in the final outcomes set.

One echocardiographer (T.V.J.), a fellow of the American Society of Echocardiography, reviewed all studies. Intraobserver variability was calculated as the correlation of the assessment of 10 cases reviewed by the same observer on different dates. Interobserver variability was calculated as the correlation of 10 cases reviewed by the study observer and another fellow of the American Society of Echocardiography (J.D.S.). The studies were classified as having correct assessments of diastolic function, incorrect assessments, or absent assessments (definitions below).

Technical Protocol

A new technical protocol for the classification of diastolic function based on a multiple-index evaluation was created. Four parameters were incorporated: (1) LV inflow, (2) e′, (3) Vp, and (4) LAVI.

LV inflow pattern was assessed by pulsed-wave Doppler positioned at the mitral leaflet tips during diastole at end-expiration. The peak E-wave velocity, peak A-wave velocity, E/A ratio, and E-wave deceleration time were measured and recorded. LV inflow pattern was characterized as normal pattern (E/A ratio > 1.0), abnormal relaxation pattern (E/A ratio < 1.0), or restrictive pattern (E/A ratio > 2.0 and E-wave deceleration time < 160 msec). A “pseudonormal” pattern was designated when a normal E/A ratio (1.0–2.0) of LV inflow was accompanied by abnormal e′, or if this was inconclusive, when Vp was abnormal.

Tissue Doppler velocity of the mitral annulus in early diastole (e′) was obtained using standard velocity presets, with pulsed spectral Doppler at the septal and lateral borders of the insertion point of the mitral leaflets in the apical four-chamber view at end-expiration. Data were obtained with an insonation angle of <20° relative to the plane of motion of the mitral annulus. Peak velocity of the early mitral annular velocity (e′ wave) was measured and recorded. Septal e′ velocity was considered abnormal if velocity was <0.08 m/sec. In cases of LV pacing or overt septal wall motion abnormality, the lateral e′ velocity was used and considered abnormal if the velocity was <0.09 m/sec. In cases of severe mitral annular calcification, previous mitral valve surgery, constrictive pericarditis, or severe mitral regurgitation e′ was excluded from the analysis. When there were regional abnormalities in systolic function or an unexplained discrepancy between the septal and the lateral annular velocities, the average of the two velocities was used. The ratio of the peak LV inflow E wave to the peak e′ wave (E/e′) was calculated and recorded. A value > 15 was interpreted to signify elevated left atrial pressure. A value of 9 to 14 was considered indeterminate, and a value < 8 was interpreted as reflecting normal left atrial pressure. The E/e′ ratio was excluded from use for the determination of left atrial pressure in cases of severe mitral regurgitation, significant annular calcification, concomitant basilar septal and lateral wall motion abnormalities, or history of mitral valve surgery.

Vp, reflective of the LV intraventricular gradient necessary for LV filling, was assessed by the slope method as previously outlined. Briefly, color flow imaging with a narrow color sector width focusing on the mitral valve in the apical four-chamber view was obtained. The M-mode scan line was then placed in the center of LV inflow. Vp was measured as the slope of the first aliasing velocity measured from the mitral annular plane to 4 cm into the LV cavity. A slope < 45 cm/sec was considered abnormal. In cases of LV hypertrophy with normal or diminished LV chamber size and normal ejection fraction, Vp may be pseudonormal. In these cases, Vp was used as part of the assessment for diastolic function only if <45 cm/sec. The ratio of the peak LV inflow E wave to Vp (E/Vp) was also recorded. A ratio > 2.5 was considered indicative of elevated left atrial pressure.

LAVI is a more accurate assessment of left atrial chamber size than traditional measures of left atrial diameter and is considered an accurate gauge for assessing diastolic function. LAVI was measured by tracing the circumferential area of the left atrium at end-systole in the apical two-chamber and four-chamber views from the back wall of the left atrium to a line across the mitral valve hinge points. Using the length of the left atrium, the volume was calculated as per Figure 1 : 8/3π × [(A1 × A2)/L)/body surface area, where A is area and L is length. LAVI values were categorized as per American Society of Echocardiography recommendations, with >28 mL/m ² as the cutoff for abnormal LAVI. In cases of long-standing atrial fibrillation, significant mitral valve disease, elite athletes, anemia, four-chamber dilation, and bradycardia, it is recognized that LAVI may not reflect diastolic dysfunction. In these situations, this parameter was not used to assess diastolic function.

Calculation of LAVI using the area-length method. Measurements were made at end-systole in the apical two-chamber and apical four-chamber views. Left atrial volume = 8/3π × [(A1 × A2)/L]/body surface area, where A is area and L is length (the shorter of L1 or L2).

Classification of Diastolic Function

The assessment of diastolic function was based on a modified Tajik scale of grading diastolic dysfunction (see Table 1 ). Diastolic function was considered normal when there was a normal LV inflow pattern with normal e′. Grade I (mild) diastolic dysfunction was considered present when LV inflow showed an abnormal relaxation pattern. In cases of tachycardia, wherein LV inflow may mimic abnormal relaxation, at least one other parameter of diastolic function was required to be abnormal (e′, Vp, or LAVI). Grade Ic (mild to moderate) diastolic dysfunction used the same criteria as grade I plus the additional criterion that evidence of elevated left atrial pressure was also present (see Figure 2 ). Signs of elevated left atrial pressure include E/e′ ratio > 15, E/Vp ratio > 2.5, E-wave velocity > 120 cm/sec, and/or atrial septal bowing rightward during systole and diastole (see Table 2 ). Grade II (moderate) diastolic dysfunction was considered present when LV inflow demonstrated a normal E/A ratio and abnormal e′ and increased LAVI (pseudonormal pattern). In cases in which e′ was excluded from analysis, abnormal values of Vp were used to determine the diagnosis. Finally, grade III (severe) diastolic dysfunction was considered present when LV inflow demonstrated a restrictive inflow pattern with abnormal e′ and LAVI.

Grade Ic diastolic dysfunction. LV inflow demonstrates abnormal relaxation pattern (A) . Decreased e′ velocity (measured by pulsed spectral Doppler) with calculated E/e′ ratio of 28 (elevated) consistent with elevated left atrial pressure (B) . Vp demonstrates a decreased slope of 40 cm/sec (C) . LAVI is increased at 36 mL/m ² (D) .

In cases of atrial fibrillation, LV inflow patterns could not be used in the manner outlined above. In addition, when atrial fibrillation is of long standing, left atrial dilation may not reflect LV filling pressures, so LAVI could not be used as part of the diagnostic criteria. In atrial fibrillation, diastolic function was considered normal when e′ was normal or, in cases in which this was inconclusive, when Vp was normal. Grade I diastolic dysfunction was considered present when e′ was abnormal or, when inconclusive, when Vp was abnormal. Grade II diastolic dysfunction was considered present when e′ was abnormal and there were changes of elevated left atrial pressure (as above; see Figure 3 ). Grade III diastolic function was considered present when the LV E wave had a decreased E-wave deceleration time of <160 msec and e′ was abnormal or, if e′ was inconclusive, when Vp was abnormal.

Grade II diastolic dysfunction with atrial fibrillation. LV inflow demonstrates increased E velocity with absent A wave (A) . Vp demonstrates a decreased slope of 36 cm/sec (B) . Decreased e′ velocity (measured by pulsed spectral Doppler) with a calculated E/e′ ratio of 23 (elevated) consistent with elevated left atrial pressure (C) . LAVI was not used as part of assessment, because of the presence of atrial fibrillation (see text) (D) .

Finally, in some cases, assessment of diastolic function was inconclusive as a result of incomplete or inadequate data acquisition. Such studies were designated as inconclusive, and the reason for this was recorded (i.e., incomplete vs inadequate data).

The use of a grading system for diastolic function in reports was not consistent at baseline, so the accuracy of assessment correlated a report grade of mild diastolic dysfunction with grade I diastolic function, a report grade of mild to moderate diastolic dysfunction with grade Ic diastolic dysfunction, a report grade of moderate diastolic dysfunction with grade II diastolic dysfunction, and a report grade of severe diastolic dysfunction with grade III diastolic dysfunction.

Classification of Assessment Accuracy

Studies were reviewed by one echocardiographer (T.V.J.), and diastolic function grade was assigned. This was then correlated with the echocardiographic report and then classified as correct if there was agreement, as incorrect if there was no agreement, and as absent if there was no mention of diastolic function in the report.

Classification of Technical Accuracy

Studies were reviewed from a technical standpoint on the basis of the completeness and accuracy of data acquisition. Studies were classified as technically incorrect if data acquisition was performed improperly (e.g., Doppler interrogation of LV inflow obtained at the annulus rather than the leaflet tips). To assess sonographer performance, studies were graded on the basis of the technical accuracy and completeness of data acquisition of all four parameters. Each parameter was classified as technically correct, incorrect, or absent.

Quality Improvement Protocol

A baseline performance set was obtained to assess the function of the echocardiography lab to accurately characterize diastolic function. This included analysis of the technical accuracy and completeness of the studies as well as the appropriate designation of diastolic function in the report.

After the baseline performance set, a plan was created to improve performance: (1) sonographer and physician education, (2) the implementation of data acquisition protocol changes and the establishment of uniform criteria for diagnostic interpretation, (3) peer review of performance, and (4) focused interactive case review sessions (see Figure 4 ).

Quality improvement protocol. Four steps were used to improve the assessment of diastolic function. A baseline survey established the function of the echocardiography lab to accurately characterize diastolic function. In step 1, an educational program for staff was given. In step 2, changes in data acquisition and reporting criteria for diastolic function were made. In step 3, a second survey with individualized performance feedback was disseminated to staff members. In step 4, a follow-up focused educational conference for staff members was held.

Step 1 involved education of the sonographers and echocardiographers. The multiple-index assessment of diastolic function outlined above was reviewed at an SHVI monthly echocardiographic teleconference for physicians and sonographers. For those unable to attend the conference or view in real time online, the presentation was archived on our internal Web site. Several focused PowerPoint (Microsoft Corporation, Redmond, WA) presentations were also posted online, further reviewing parameters for diastolic function assessment and the criteria for classification of diastolic function.

Step 2 involved communicating protocol changes for data acquisition and reporting via e-mail to all SHVI sonographers and echocardiographers. Modifications were made to the phrase files in the SHVI echocardiographic report templates to reduce variance in echocardiographer description of diastolic function. An analysis support tool was included in the report templates to guide correct reporting of diastolic function (see the Appendix ).

After allowing an initial 3-month period for adjustments to the new protocol, in step 3, the peer review survey (post–educational intervention set) was completed. As outlined above, the studies were reviewed for technical completeness and accuracy as well as diagnostic interpretative accuracy. Each sonographer then received feedback via a report card identifying his or her performance, along with how that performance compared with that of the other sonographers. Similarly, each echocardiographer also received a report card benchmarking absolute and relative performance in the accuracy of diagnostic interpretation.

In step 4, on the basis of results of the peer review survey, a focused case study and interactive review session was held at the SHVI echocardiography teleconference. Common issues, as identified by the peer review survey, involving the technical acquisition and misinterpretation of diastolic function were reviewed. Audience participation in the session was considered important. Each case was presented, and the audience was asked to critique the studies’ technical aspects as well as classify the diastolic function.

Assessment of Improvement

Approximately 3 months after the focused case review and interactive review session, a second survey was performed to track improvement in diagnostic imaging and interpretation of diastolic function (final outcomes set). Methodologies for the evaluation of the final outcomes set were identical to those outlined for the baseline performance set.

Statistical Analysis

Chi-square tests were performed to compare baseline, post–educational intervention, and final outcomes accuracy of diastolic function evaluations. All variables were summarized using frequencies and percentages. P values < .05 were considered to indicate statistical significance. A random sample of 10 evaluations was taken to determine interrater and intrarater reliability. Cohen’s κ coefficient was used to further assess interrater agreement at baseline and after the final survey. All analyses were conducted using SAS version 9.2 (SAS Institute Inc., Cary, NC). Intrarater reliability was determined to be 100% for the random sample: all 10 sample assessments had the same diastolic function between time points for the same rater. Interrater reliability was 90%: nine of the 10 assessments had the same diastolic function when taken between two different raters. The κ statistic for the baseline survey was 0.48 (95% confidence interval, 0.27–0.70), which indicates moderate agreement, while the κ statistic for the final survey was found to be 0.74 (95% confidence interval, 0.63–0.85), which indicates substantial agreement. It should be noted that certain categories, which did not have ratings in both groups, were excluded to compute the κ statistics.