Background
Accurate visual assessment of wall motion during dobutamine stress echocardiography (DSE) requires expertise; strain stress echocardiography thus has potential for the objective and reliable evaluation of stress-induced wall motion abnormalities. The aim of this study was to test the hypothesis that strain imaging also requires expertise. Diagnostic accuracy for significant coronary artery disease was compared between visual and strain analysis with a bull’s-eye map, and the effect of expertise on readers’ accuracy during DSE was determined.
Methods
Thirty-seven patients who underwent both DSE and coronary angiography within 6 months were retrospectively selected. Three experts and two fellows performed visual and two-dimensional speckle-tracking analysis using vendor-dependent (GE) software in three apical views. A bull’s-eye map of longitudinal strain and a postsystolic shortening map were generated at baseline and peak stress. Significant coronary stenosis was defined as a >1% reduction in longitudinal strain with an increase in postsystolic shortening of at least two contiguous segments in the specific coronary artery territory at peak stress compared with baseline.
Results
Twenty-five patients had significant coronary artery disease (>70% stenosis on coronary angiography), including 17 with left anterior descending coronary artery stenosis and 19 with non–left anterior descending coronary artery stenosis. Overall, strain imaging provided no additional benefit over visual analysis in detecting significant stenosis per patient and per vessel among experts and fellows. Strain analysis by the fellows had significantly lower specificity and accuracy compared with the experts.
Conclusions
Expertise affects strain evaluation for detecting significant coronary artery disease during DSE.
Highlights
- •
The authors hypothesized that expertise affects not only visual assessment but also strain analysis during DSE.
- •
The authors investigated the diagnostic accuracy for CAD of 2D strain analysis using bull’s-eye and postsystolic shortening maps on DSE among experts and fellows.
- •
Two-dimensional strain analysis provided no additional value in diagnostic accuracy compared with visual analysis among both experts and fellows.
- •
Two-dimensional strain analysis was expertise dependent. Fellows had lower specificity, resulting in lower diagnostic accuracy, compared with experts.
- •
Expertise affects not only visual wall motion estimation but also 2D strain analysis for detecting significant CAD.
Stress echocardiography has been the method of choice in diagnosing significant coronary artery disease (CAD) for more than two decades. Because of an increase in the older population, dobutamine stress echocardiography (DSE) is the best modality for CAD detection in patients who cannot exercise adequately. The visual estimation of stress-induced transient regional wall motion abnormalities is subjective and requires expertise to achieve high diagnostic accuracy. Disagreement regarding diagnostic results has been reported, even among experts in an interinstitutional study. Following the advent of tissue Doppler and two-dimensional (2D) speckle-tracking analysis, several investigators have attempted to apply deformation imaging to obtain quantitative information that could possibly enhance diagnostic accuracy. Investigators have usually used the change in strain and/or strain rate from baseline to peak stress or recovery on a global or regional basis to detect significant CAD. However, a majority of studies determined their optimal cutoff values using receiver operating characteristic curve analysis. Thus, there is currently no consensus regarding which parameter and which cutoff value is most appropriate for high diagnostic accuracy. Although a bull’s-eye map of longitudinal strain (LS) provides one-look recognition of the severity and extent of longitudinal dysfunction, no studies have determined its potential utility during DSE. Furthermore, we hypothesized that strain measurements in 2D speckle-tracking analysis also requires expertise for accurate and reliable analysis. In other words, the accuracy of both visual assessment and 2D strain analysis depends heavily on experience, which may lead to different results depending on the physician’s degree of expertise. Although prior studies have demonstrated the advantages of strain imaging in DSE, the effect of expertise on the diagnostic accuracy has not been determined.
Accordingly, the aims of this study were (1) to compare diagnostic accuracy between visual analysis and strain assessment using a bull’s-eye map for detecting significant CAD and (2) to investigate the effect of expertise (between expert and fellow physicians) on the diagnostic accuracy of strain analysis compared with visual assessment during DSE.
Methods
Study Patients
We retrospectively screened patients in sinus rhythm who underwent clinically indicated DSE for the evaluation of myocardial ischemia and subsequently underwent invasive diagnostic coronary angiography within 6 months. Among the 171 consecutive dobutamine stress echocardiographic examinations performed during the study period, 134 were performed solely for the evaluation of valvular heart disease or had no concomitant coronary angiography. The remaining 37 cases fulfilled the inclusion criteria and formed this study group. None were excluded because of poor image quality. The hospital’s ethics committee approved the study protocol, and the requirement to obtain informed consent was waived because of the retrospective nature of the study. Five examiners participated, including three experts and two fellows. The experience of the experts was >20, 10, and 6 years in the performance and interpretation of stress echocardiography, as well as 10, 5, and 3 years in 2D strain analysis. The fellows had completed level 3 training, including the performance of ≥300 transthoracic echocardiographic studies and the interpretation of ≥750 transthoracic echocardiograms and ≥80 stress echocardiograms (exercise or dobutamine). As for strain analysis, the fellows had analyzed ≥300 cases with training in both strain curves and bull’s-eye displays. Training includes confirmation of the tracking quality from the display and adjustment of preprocessing endocardial tracking whenever necessary. Fellows were also trained to determine postsystolic shortening (PSS) on each strain curve. These examiners individually performed both the visual assessment and 2D strain analysis with speckle-tracking software during DSE.
DSE
DSE was performed using a standard protocol. Dobutamine was administered at a starting dose of 5 μg/kg/min after recording the baseline echocardiogram. The dose was then increased in 3-min intervals up to 40 μg/kg/min. Intravenous atropine at a dose of 0.5 mg (up to 1 mg) was given if no end points were observed at 40 μg/kg/min of dobutamine infusion. End points of DSE were the achievement of target heart rate (>85% of age-predicted maximal heart rate), completion of the stress protocol, or obvious new-onset or worsening of wall motion abnormalities. Two-dimensional stress echocardiographic imaging was performed by experienced sonographers or physicians using the GE Vivid 7 and M4S transducer (GE Vingmed Ultrasound AS, Horten, Norway) while patients were lying in the left decubitus position. Standard parasternal long- and short-axis views and three apical views at frame rates ranging from 60 to 100 frames/sec were obtained at rest, each stage of dobutamine infusion, and recovery. Images were stored as cine loops in three consecutive beats for offline visual assessment as well as 2D speckle-tracking analysis.
Visual Assessment of Wall Motion
For visual assessment, we used only three apical views for the sake of consistency and a direct comparison between the visual and strain analysis. The left ventricle was divided into 18 segments, and wall motion and thickness in each segment were evaluated in a side-by-side cine-loop comparison between the rest and peak stress images. Stress-induced myocardial ischemia was defined as new or worsening wall motion abnormalities induced by dobutamine stress in at least two contiguous left ventricular segments in the same coronary territory. Although we did not select patients on the basis of the image quality, none of the patients were excluded because of incapability in visual diagnosis. The left ventricle was divided into anterior circulation, which corresponds to the perfusion territory of the left anterior descending coronary artery (LAD), and posterior circulation, which is supplied by the right coronary artery (RCA) or left circumflex coronary artery (LCX) to determine individual coronary artery stenosis (LAD or non-LAD).
Two-Dimensional Speckle-Tracking Echocardiography
Quantitative strain analysis using vendor-dependent software (EchoPAC PC version 201; GE Vingmed Ultrasound AS) was performed individually by both the experts and the fellows. None of the physicians knew any clinical information of the patients, with the exception of their age and sex. After manual endocardial and epicardial border tracing in an end-systolic frame in the three apical views, the region of interest was generated, which was edited whenever necessary for better border delineation and better coverage of the full thickness of the myocardium. The software performed speckle-tracking analysis during one cardiac cycle to generate a strain curve in each segment. If there were any segments with persistent poor tracking after several attempts at border editing, we excluded the image from the analysis. The timing of aortic valve closure was visually defined using the apical long-axis view. Finally, we recorded the global LS (GLS), the bull’s-eye map of the peak systolic LS, and the map of the PSS index (PSI) at both baseline and peak stress. PSS is the further longitudinal contraction after aortic valve closure and a sign of ischemia. PSI is defined as follows: ([peak LS] − [end-systolic LS])/peak LS. A decrease in myocardial strain during systole in conjunction with PSS has been reported to be a sensitive parameter for myocardial ischemia. Therefore, we defined myocardial ischemia as a >1% absolute reduction in the segmental LS in more than two contiguous segments within the specific coronary artery territory on an end-systolic LS bull’s eye map and newly appearing PSS or exaggerated PSS on the PSI map on the corresponding segments at peak stress ( Figure 1 ).
To detect significant coronary stenosis in the major coronary arteries, we divided the left ventricle into anterior circulation (LAD territory: basal to middle anteroseptal and anterior wall, middle interventricular septum, and apex) and posterior circulation (RCA and LCX territory) because the posterior wall overlaps the region between the RCA and LCX. Significant coronary artery stenosis in the LAD and RCA or LCX (non-LAD) were determined in a specific coronary artery territory by comparing the change between the resting and peak stress imaging using the same criteria described above.
Coronary Angiography
The images from diagnostic coronary angiography were interpreted by an experienced interventional cardiologist. Significant CAD was defined as >70% luminal diameter stenosis in at least one of the three major coronary arteries measured in the worst-view angiographic projection. Because of retrospective nature of the selection of the patients, intravascular ultrasound or fractional flow reserve measurement was not systematically conducted in this study.
Interobserver Variability
Interobserver variability in GLS measurements between each paired observer (expert vs expert, expert vs fellow, and fellow vs fellow) was represented using Pearson correlation, bias, limit of agreement, intraclass correlation coefficient, and percentage variability. Interobserver variability in diagnosis on a patient basis and a vessel basis was estimated using a concordance rate with κ statistics.
Statistical Analysis
Continuous data are expressed as mean ± SD or as median (interquartile range) according to the data distribution. Normality was evaluated using the Shapiro-Wilk W test. Categorical data are presented as numbers or percentages. All statistical analyses were performed using JMP version 11.0 (SAS Institute, Cary, NC), Prism version 5.0 (GraphPad Software, San Diego, CA), or SPSS version 18.0 (SPSS, Chicago, IL). To avoid unnecessary confusion, we report the absolute values of GLS in the text, figures, and tables, in accordance with definitions from the standardization task force, with the exception of the original bull’s-eye map. GLS between baseline and peak stress was compared using the Wilcoxon signed rank test. GLS results among five observers were compared using the nonparametric Friedman test with post hoc analysis done between two paired observers. Sensitivity, specificity, and accuracy were calculated using standard formulae. Accuracy was calculated as the total number of true positive and true negative tests divided by the total number of patients. The statistical differences between expert and fellow as well as between visual assessment and strain analysis for individual observer were analyzed by chi-square or Fisher’s exact analysis. P < .05 was considered statistically significant.
Results
Among the 37 patients (mean age, 73 ± 11 years; 43% women) included in our study, 68% had significant CAD, 41% had multivessel disease, and 38% had baseline wall motion abnormalities ( Table 1 ). End point distribution was as follows: achievement of the target heart rate in 22 patients, end of the protocol in nine, new or worsening wall motion abnormality in four, and others in two. Atropine was given to 11 patients (30%). The frame rate between rest and peak stress was similar (66 vs 66 frames/sec, P = .3240).
| Variable | n = 37 |
|---|---|
| Age (y) | 73.4 ± 10.9 |
| Women | 16 (43%) |
| Body surface area (m 2 ) | 1.55 ± 0.19 |
| Hyperlipidemia | 19 (51%) |
| Hypertension | 24 (65%) |
| Diabetes mellitus | 16 (43%) |
| Current or previous smoking | 9 (24%) |
| Old myocardial infarction | 10 (27%) |
| LV hypertrophy | 23 (62%) |
| β-blocker | 12 (32%) |
| Calcium channel blocker | 7 (19%) |
| Nitrates | 2 (5%) |
| LVEF (%) | 68 (52.5–76.5) |
| LV mass index (g/m 2 ) | 117.9 (92.8–135.2) |
| Baseline wall motion abnormality | 14 (38%) |
| Coronary angiography | |
| Coronary artery disease | 25 (68%) |
| No significant stenosis | 12 (24%) |
| One-vessel disease | 10 (27%) |
| Multivessel disease | 15 (41%) |
| LAD affected | 17 (46%) |
| Hemodynamics | |
| Baseline systolic blood pressure (mm Hg) | 155.6 ± 25.7 |
| Peak systolic blood pressure (mm Hg) | 162.4 ± 37.2 |
| Baseline heart rate (beats/min) | 67.6 ± 13.3 |
| Peak heart rate (beats/min) | 119.6 ± 25.5 |
| Percentage predicted heart rate achieved | 86.1 ± 14 |
Of the 111 analyzable apical images (three images per patient), none were excluded because of persistent poor tracking. Visual analysis had 100% feasibility. GE software could not provide a bull’s-eye map for either baseline or peak stress in zero to three patients for the expert physicians and zero to four patients for the fellow physicians, primarily because of the differing heart rate (sinus arrhythmia) in the three apical views.
Diagnostic Accuracy of Visual and Strain Analysis
Table 2 presents the sensitivity, specificity, and accuracy among the five observers for diagnosing >70% coronary stenosis. On a patient basis, the expert had 59% to 70% diagnostic accuracy in visual assessment, which did not significantly improve using strain analysis, because strain imaging yielded higher sensitivity but lower specificity than visual analysis ( Table 2 ). For LAD stenosis, diagnostic accuracy was similar between visual assessment and strain analysis among experts. Strain analysis had slightly lower diagnostic accuracy compared with visual assessment for detecting significant RCA and LCX stenosis.
| Sensitivity | Specificity | Accuracy | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Expert 1 | Expert 2 | Expert 3 | Fellow 1 | Fellow 2 | Expert 1 | Expert 2 | Expert 3 | Fellow 1 | Fellow 2 | Expert 1 | Expert 2 | Expert 3 | Fellow 1 | Fellow 2 | |
| Visual assessment | |||||||||||||||
| Patient basis | 63% (12/19) | 68% (13/19) | 78% (15/19) | 79% (15/19) | 63% (12/19) | 78% (14/18) | 67% (12/18) | 39% (7/18) | 56% (10/18) | 61% (11/18) | 70% (26/37) | 68% (25/37) | 59% (22/37) | 68% (25/37) | 62% (23/37) |
| LAD | 56% (6/11) | 45% (5/11) | 73% (8/11) | 56% (6/11) | 64% (7/11) | 88% (23/26) | 85% (22/26) | 69% (18/26) | 88% (23/26) | 88% (23/26) | 78% (29/37) | 73% (27/37) | 70% (26/37) | 78% (29/37) | 81% (30/37) |
| Posterior circulation | 43% (6/14) | 50% (7/14) | 71% (10/14) | 79% (11/14) | 57% (8/14) | 87% (20/23) | 83% (19/23) | 70% (16/23) | 61% (14/23) | 74% (17/23) | 70% (26/37) | 70% (26/37) | 70% (26/37) | 68% (25/37) | 68% (25/37) |
| Strain imaging | |||||||||||||||
| Patient basis | 78% (14/18) | 65% (11/17) | 89% (17/19) | 89% (17/19) | 82% (14/17) | 44% (7/16) | 53% (9/17) | 50% (9/18) | 22% (4/18) | 13% † , ‡ , § (2/16) | 62% (21/34) | 59% (20/34) | 70% (26/37) | 57% (21/37) | 48% (16/33) |
| LAD | 64% (7/11) | 70% (7/10) | 91% (10/11) | 73% (8/11) | 70% (7/10) | 83% (19/23) | 79% (19/24) | 65% (17/26) | 46% ∗ , † , § (12/26) | 39% ∗ , † , § (9/23) | 76% (26/34) | 76% (26/34) | 73% (27/37) | 54% § (20/37) | 48% ∗ , † , ‡ , § (16/33) |
| Posterior circulation | 64% (9/14) | 43% (6/14) | 79% (11/14) | 64% (9/14) | 77% (10/13) | 50% § (10/20) | 80% (16/20) | 61% (14/23) | 30% † (7/23) | 15% ∗ , † , ‡ , § (3/20) | 56% (19/34) | 65% (22/34) | 68% (25/37) | 43% (16/37) | 39% ‡ , § (13/33) |
∗ P < .05, expert 1 versus fellow 1 or fellow 2.
† P < .05, expert 2 versus fellow 1 or fellow 2.
‡ P < .05, expert 3 versus fellow 1 or fellow 2.
§ P < .05, visual assessment versus strain analysis for the same observer.
As for the fellow physicians, diagnostic accuracy in visual assessment ranged from 62% to 68% on a patient basis. Diagnostic accuracy of strain imaging was lower than that of visual analysis, predominantly because of its very low specificity ( Table 2 ). On a vessel basis, visual assessment among fellow physicians had significantly higher diagnostic accuracy than strain assessment, because lower specificity predominated the marginal increase in sensitivity. The subgroup analysis in patients without regional wall motion abnormalities showed similar results ( Supplemental Table 1 ).
Diagnostic Accuracy between the Expert and Fellow Physicians
The diagnostic accuracy of visual assessment for significant CAD on a patient basis as well as a vessel basis by the experts was not significantly different from that of the fellows ( Table 2 ). However, for strain imaging, diagnostic accuracy was better among experts than among fellows (59%–70% vs 48%–57% for the patient, 73%–76% vs 48%–54% for the LAD stenosis, and 56%–68% vs 39%–43% for the non-LAD stenosis, respectively), which was due to lower specificity among fellows than among experts ( Table 2 ).
Tables 3 A and 3B depicts interobserver differences in GLS among the five examiners. The mean value of GLS was significantly different among the five examiners at baseline as well as at peak stress. GLS was significantly increased from baseline to peak stress in two experts. However, no significant differences were noted in one expert and two fellows. At baseline, the mean value of bias, limits of agreement, and percentage variability of GLS were lowest when the comparison was made between expert and expert (0.62, ±2.00, and 10.5%, respectively), followed by the comparison between expert and fellow (1.16, ±2.40, and 12.2%, respectively), and highest in the comparison between fellow and fellow (2.38, ±2.72, and 17.2%, respectively). The intraclass correlation coefficient was also highest between expert and expert (0.958), followed by between expert and fellow (0.914), and least between fellow and fellow (0.823). The same trend was also observed at peak stress. Specifically, the mean value of bias, limits of agreement, and percentage variability of GLS were lowest (1.36, ±2.99, and 13.6%, respectively) and intraclass correlation coefficient was highest (0.898) when the comparison was made between expert and expert. The worst values (2.86, ±3.68, 19.5%, and 0.761, respectively) were observed in the comparison between fellow and fellow. These findings implied that not only visual assessment but also strain analysis require training to obtain accurate and consistent results.
| Expert 1 | Expert 2 | Expert 3 | Fellow 1 | Fellow 2 | ANOVA | |
|---|---|---|---|---|---|---|
| Baseline | 14.4 (11.1–16.3) | 15.0 (11.4–17.0) ∗ | 14.4 (10.3–16.4) † | 13.2 (10.7–15.9) † | 16.9 (12.6–18.1) ∗ , † , ‡ , § | <.001 |
| Peak stress | 14.3 (11.0–17.2) | 17.0 (12.5–19.5) ∗ | 14.5 (11.3–17.5) † | 13.8 (9.6–16.6) † | 17.0 (12.2–20.1) ∗ , ‡ , § | <.001 |
| P | .2447 | .0038 | .0260 | .9326 | .4689 |
∗ P < .05, post hoc analysis, expert 1 versus expert 2, expert 3, fellow 1, or fellow 2 for baseline and peak GLS.
† P < .05, post hoc analysis, expert 2 versus expert 3, fellow 1, or fellow 2 for baseline and peak GLS.
‡ P < .05, post hoc analysis, expert 3 versus fellow 1 or fellow 2 for baseline and peak GLS.
§ P < .05, post hoc analysis, fellow 1 versus fellow 2 for baseline and peak GLS.
| Baseline GLS | Peak stress GLS | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| r | Bias | LOA | % variability | ICC | r | Bias | LOA | % variability | ICC | |
| Expert 1 vs expert 2 | 0.97 | −0.93 | ±2.09 | 12.7 | 0.950 | 0.94 | −2.00 | ±3.37 | 18.3 | 0.855 |
| Expert 1 vs expert 3 | 0.96 | −0.01 | ±2.17 | 12.2 | 0.966 | 0.93 | −0.63 | ±3.07 | 12.2 | 0.925 |
| Expert 2 vs expert 3 | 0.98 | 0.91 | ±1.74 | 6.7 | 0.960 | 0.96 | 1.45 | ±2.54 | 10.4 | 0.914 |
| Expert 1 vs fellow 1 | 0.96 | 0.23 | ±2.25 | 12.2 | 0.960 | 0.90 | 0.64 | ±3.86 | 16.8 | 0.894 |
| Expert 1 vs fellow 2 | 0.97 | −2.08 | ±2.17 | 18.8 | 0.868 | 0.85 | −2.05 | ±5.17 | 18.0 | 0.735 |
| Expert 2 vs fellow 1 | 0.94 | 1.16 | ±2.97 | 8.9 | 0.907 | 0.94 | 2.49 | ±3.25 | 19.2 | 0.811 |
| Expert 2 vs fellow 2 | 0.98 | −1.18 | ±1.76 | 9.8 | 0.950 | 0.95 | −0.48 | ±3.35 | 8.5 | 0.926 |
| Expert 3 vs fellow 1 | 0.93 | 0.22 | ±2.95 | 7.6 | 0.934 | 0.84 | 1.43 | ±4.7 | 16.5 | 0.802 |
| Expert 3 vs fellow 2 | 0.96 | −2.11 | ±2.35 | 15.8 | 0.865 | 0.84 | −1.70 | ±5.15 | 17.1 | 0.780 |
| Fellow 1 vs fellow 2 | 0.95 | −2.38 | ±2.72 | 17.2 | 0.823 | 0.93 | −2.86 | ±3.68 | 19.5 | 0.761 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
