Background
Three-dimensional (3D) echocardiography is fundamental for left ventricular (LV) assessment. The aim of this study was to determine discrepancies in 3D LV endocardial tracings and suggest tracing guidance.
Methods
Forty-five 3D LV echocardiographic data sets were traced by three experienced operators, from different centers, according to predefined guidelines. The 3D meshes were compared with one another, and the endocardial areas of discrepancies were identified. A discussion and retracing protocol was used to reduce discrepancies. For each data set, an average 3D mesh was produced (reference mesh). Subsequently, four novice operators, divided into two groups, traced 20 of the data sets. Two operators followed the tracing protocol and two did not.
Results
The intraclass correlation coefficients among the three experienced operators for end-diastolic volume, end-systolic volume, and ejection fraction were 0.952, 0.955, and 0.932. The absolute distances between tracings were 1.11 ± 0.45 mm. The highest tracing discrepancies were at the apical cap and anterior and anterolateral walls in end-diastole and end-systole and also at the basal anteroseptum in end-systole. Agreement with the reference meshes was better for the novice operators who followed the guidance (10.9 ± 17.3 mL, 10.2 ± 14.7 mL, and −2.2 ± 4.1% for end-diastolic volume, end-systolic volume, and ejection fraction) compared with those who did not (16.3 ± 16.4 mL, 17.0 ± 16.0 mL, and −4.2 ± 4.1%, respectively).
Conclusions
Comparing 3D LV tracings, the endocardial areas that are the most difficult to delineate were identified. The suggested protocol for LV tracing resulted in very good agreement among operators. The reference 3D meshes are available for online testing and ranking of LV tracing algorithms.
Graphical abstract
Highlights
- •
The authors propose a protocol for 3D ultrasound LV endocardial tracing.
- •
By direct comparison of three operators’ tracings, the areas of highest distance errors were identified.
- •
The authors provide reference 3D LV meshes for evaluation of tracing algorithms.
- •
The suggested tracing protocol can improve the accuracy of novice operators’ tracings.
Three-dimensional (3D) echocardiography provides significant advantages over two-dimensional (2D) echocardiography and is currently applied in several aspects of cardiology. The most common indication for performing echocardiography in adults is the evaluation of left ventricular (LV) size and function. The use of 3D echocardiographic imaging eliminates geometric assumptions and misinterpretation errors caused by foreshortened views in 2D mode. Several trials have demonstrated the reproducibility of 3D-derived LV measurements. At present there are no clear standards or guidelines available for 3D LV endocardial border tracing, and there is no direct comparison of actual tracings among different operators.
Automated tracing of the left ventricle in 3D cardiac ultrasound data sets has been a subject of scientific research for the past 20 years, but there has hardly been any comparison of different methods on the same data sets.
In this study, we aimed to address these issues by suggesting a protocol for LV endocardial tracing in 3D echocardiographic data sets and creating a series of clinically realistic data sets with well-established reference tracings on the basis of manual tracings from three expert echocardiography centers. On the basis of this standard set, a competition for automated tracing methods was organized, associated with the Medical Image Computing and Computed Assisted Interventions 2014 symposium, which has been published previously. The purpose of this competition was to provide reference 3D LV meshes for testing LV endocardial tracing algorithms. The reference meshes remain available online for continuous testing and ranking of fully automated or semiautomated algorithms.
Finally, we evaluated the usefulness of our tracing protocol in a clinically relevant setting, in which commercially available software was used by novice operators.
Methods
Acquisition Protocol
We included 45 individuals: 15 healthy individuals, 15 patients with previous myocardial infarction at least 3 months before the time of echocardiography, and 15 patients with nonischemic dilated cardiomyopathy. The patients were recruited at three different institutions (Rennes University Hospital, Rennes, France; University Hospital Leuven, Leuven, Belgium; and Thoraxcenter, Erasmus MC, Rotterdam, The Netherlands). Fifteen patients undergoing echocardiography and meeting the inclusion criteria were recruited at each institution. Exclusion criteria were left bundle branch block, visually dyssynchronous left ventricle, and unacceptable image quality. Unacceptable image quality was defined as (1) significant stitching or other types of artifacts affecting the tracking of endocardium or (2) poor visualization of the LV wall or wall out of the image sector to an extent that the image could no longer be manually analyzed with good confidence in multiple segments. The image quality of the accepted data sets was graded as good, fair, or poor ( Figure 1 ). Good quality was defined when the endocardium was visible in end-diastolic (ED) and end-systolic (ES) instances in all 17 segments throughout the cardiac cycle, fair quality if the endocardium was not clearly seen in one or two segments, and poor quality when the endocardial border was not clearly distinguished in ES or ED frames in more than two segments, but the operator could still define the border with confidence by tracking the endocardium throughout the cardiac cycle and also by considering adjacent segments. The variation in image quality was a result of recruitment of cases in a real-life setting and was not intentional. The image quality variation was similar in all three hospitals’ data sets.
We used echocardiography machines from three different vendors: Vivid E9 with a 4V probe (GE Vingmed Ultrasound, Horten, Norway), iE33 with an X3-1 or X5-1 probe (Philips Medical Systems, Andover, MA), and SC2000 with a 4Z1c probe (Siemens Healthcare, Erlangen, Germany). Machine settings were optimized to achieve maximum image quality while keeping the volume rate above 16 Hz. The mean number of frames per cardiac cycle was 25.7 ± 8.5. Acquired data were fully anonymized and handled within the regulations set by the local ethical committee of each hospital.
Endocardial Tracing Procedure
ED and ES frames were identified. Nine standard anatomic planes were defined: four longitudinal planes through the long axis under 45° angles and five transverse (short-axis) planes divided equally along the long axis. For the tracings, a custom noncommercial tracing package for 3D echocardiograms (Speqle3D) was used, developed at the University of Leuven. A single experienced operator from each center (A.P., M.L.G., E.G.) with experience of >300 3D LV tracing analyses was appointed to perform the tracings. Each operator independently traced the endocardial border in the nine predefined planes, in both ED and ES instances. To guarantee direct comparisons, the operators were only allowed to contour in the nine predefined slices and in the allocated ED and ES frames. All 45 data sets were traced by all three operators.
A set of guidelines for performing the LV tracing was defined at the beginning of the project and revised subsequently by comparing the tracing conventions of the different centers. Basic aims were as follows:
Include trabeculae and papillary muscles in the LV cavity ( Figure 2 ). A suggestion was for the operator to take as a reference point the endocardial border that is free of papillary muscle and then trace “outside” the papillary muscle to meet the endocardium at the other edge of the muscle (i.e., from the basal to the apical segment or vice versa). Also, we suggested tracing at the level of the trough of endocardial creases to include trabeculations in the LV cavity.
Figure 2
Example of tracing including trabeculae and papillary muscles ( yellow arrows ) in the LV cavity.
Keep tissue consistency between end-diastole and end-systole and between adjacent and intersecting planes. The operator was asked to play the cine loop forward and backward to ensure that the traced endocardial border in end-diastole was corresponding to the same tissue line in end-systole by tracking the endocardium throughout the cardiac cycle. During this process, special consideration was taken with regard to elevation plane artifacts. Also, the prototype software showed the projection of the intersection points between tracings of orthogonal planes ( Figure 3 ). The operator therefore ensured tissue consistency between transverse and longitudinal planes.
Figure 3
Example of manual endocardial drawing in Speqle3D software. ( Left ) Transverse plane (short axis, SAx). ( Right ) Longitudinal plane (long axis, LAx). The green dots in both images represent the points set on the endocardial border by the operator on the actual plane. The red line represents the endocardial contour created by b-spline interpolation of the green dots . The pink dots represent the cross-sectional points of the contours in the orthogonal planes.
In long-axis views, draw up to the mitral valve annulus on the inside of the bright ridge up to the point at which the valve leaflet is hinging. The mitral valve annulus is sometimes quite difficult to trace with consistency. For this reason, we suggested that the operator should trace at the ventricular side of the annulus and pay special attention to identify the leaflet hinge point by reviewing the cine loop instead of judging on the basis of a single frame ( Figure 4 ).
Figure 4
Example of manual tracing pointing out the mitral valve (MV) hinge points ( yellow arrows ).
Partly exclude the LV outflow tract (LVOT) from the cavity by drawing from the septal mitral valve hinge point to the septal wall to create a smooth shape ( Figure 5 ). The LVOT is one of the most challenging parts of LV endocardial tracing. We proposed to trace in a way that partially excludes the LVOT and provides a smooth shape of the basal anteroseptal wall segment to keep the LV shape symmetric and also avoid giving the impression of a dyskinetic segment as the LVOT expands during systole.
Figure 5
Superimposition of the three experienced operators’ manual tracings in a long-axis view. ( Left ) End-diastole. ( Right ) End-systole. The tracing of the LVOT (bottom right end of contours) has been performed in a way that partially excludes LVOT, and a smooth curved line is drawn from the mitral valve hinge point to the septal wall curve.
Draw the apex high up near the epicardium in both end-diastole and end-systole, taking into consideration that there should be little displacement of the true apex point. As is known from anatomy, the LV wall is actually thin at the true apex and hardly moves during the cardiac cycle. Apparent apical wall thickness and motion in cross-sectional images is due to foreshortening and/or blurred trabeculations ( Figure 6 ). Also, the endocardium is more difficult to visualize compared with the epicardium at the apical region. Therefore, tracing close to epicardium can provide better anatomic ED and ES consistency than the less “visible” endocardium. This is expected to have little effect on volumes but makes the contouring more consistent.
Figure 6
Example of tracing of the apex ( yellow arrows ), which is close to epicardium with little displacement in end-systole.
Evaluation of Correspondence
After all three experts had submitted their tracings, 3D shapes ( Figure 7 ) were generated from the nine 2D contours in end-diastole and end-systole by interpolation. The shapes were represented by a 3D mesh with a resolution of about 40 by 80 points (longitudinal × circumferential). Each 3D mesh consisted of about 3,200 points (vertices). These vertices from the three operators’ tracings were averaged to produce the reference mesh. The 3D meshes of the three operators were compared pairwise and mean absolute distances, the maximal perpendicular distance between all points of two meshes (the so-called Hausdorff distance ), LV volumes, and LV ejection fraction (LVEF) differences were calculated. The surface distance between the surface of an operator’s 3D mesh and the surface of the reference mesh was represented in a 3D model using a color-coded approach ( Figure 7 ). This allowed visualization of endocardial border discrepancies between a single operator’s tracing and the reference mesh. For the purposes of the workshop, the meshes of the three operators had to be consistent with one another. To verify consensus, the following criteria were used: Hausdorff distance ≤ 5 mm between each individual mesh and the reference mesh, percentage (relative) difference in LV volumes ≤ 10% among all three operators in pairs, and absolute difference in LVEF ≤ 5 percentage points among all three operators in pairs. The percentage difference was calculated as the absolute value of the difference between two operators divided by the mean. If consensus criteria were not met, the tracings would be discussed among the operators. This was done by superimposing the tracings of the three operators for each one of the nine predefined planes at both end-diastole and end-systole ( Figure 5 ). For the purposes of our workshop, one or more of the operators would then revise their tracings to reach consensus following discussion. Then the evaluation process would be repeated and slightly milder consensus criteria were applied: the average of the three pairwise observer differences was evaluated and Hausdorff distances ≤ 7 mm were accepted. The final averaged 3D mesh for each case, after revision process, was considered the “reference mesh.”
The Tracing Guidance Applied to Novice Operators Using a Commercially Available Semiautomated Software
We additionally explored the usefulness of our tracing protocol using a vendor-independent commercially available software for 3D LV endocardial tracing (4D LV Analysis Image Arena version 3.5; TomTec Imaging Systems, Unterschleissheim, Germany). Four operators with little experience in 3D echocardiographic LV tracing were asked to trace 20 of our study data sets, which were randomly selected from the GE and Siemens data sets. The Philips data sets could not be imported to the commercially available software for analysis, because of technical limitations related to acquisition and storage process.
All four operators had experience of 20 to 30 cases and were from a single center (King’s College Hospital, London, United Kingdom). They were divided into two groups. Operators A1 and A2 (group A) traced according to their own discretion. Operators B1 and B2 (group B) were provided with our tracing protocol and recommendations. All operators were blinded to other operators’ results and also to the reference meshes.
In the TomTec platform, the software initially detects the ED and ES instances. The operator is required to define the LVOT in a short-axis plane as a reference point. From that reference point, the software identifies the apical three-chamber view and then automatically defines the two-chamber and four-chamber views (60° and 120° incremental views). The operator aligns the longitudinal axis of the left ventricle in all three apical views such that the axis crosses the LV apex and the middle of the mitral annular plane in all views. Two markers at the two ends of the long-axis line are placed by the operator at the apex and the mitral annular level in end-diastole. Subsequently, the application automatically defines the endocardial border in all three apical views in end-diastole. The operator can adjust the tracings manually as necessary in all views. Then the software tracks the endocardium throughout the cardiac cycle. Once this is completed, the operator can modify the tracings in ES and ED instances in all apical views (two-, three-, and four-chamber views) and also in a short-axis plane, which can be manually swept along the full length of the LV long axis. The operator is not allowed to modify the endocardial tracing in frames other than the ED and ES frames. The application continuously calculates the ED volume (EDV), ES volume (ESV) and LVEF as the operator manipulates the tracings. For the purposes of our study, once the final tracings were confirmed, the derived volumes and LVEF were recorded by the operator in a separate spreadsheet.
Statistical Analysis
Statistical analysis was performed using SPSS version 20.0.0 (IBM, Somers, NY). Differences in mean values for continuous variables were tested using Student’s t test, the Mann-Whitney U test for unpaired samples, or the Wilcoxon signed rank test for paired samples as appropriate. Interobserver variability was tested using the intraclass correlation coefficient (ICC) for absolute agreement in a two-way mixed model. Values for continuous variables are presented as mean ± SD. Statistical significance was considered for a two-tailed P value < .05.
Results
Baseline Demographics and Characteristics of Data Sets
The mean age of the tested population was 53.9 ± 18.8 years, and 88.9% of patients were men. The mean values and SDs of volumes and LVEF of the reference meshes are shown in Table 1 . Fourteen cases (31%) were of good quality, 16 (36%) of fair quality, and 15 (33%) of poor quality.
| Variable | Reference mesh, mean ± SD | ICC (95% CI) |
|---|---|---|
| EDV (mL) | 173.5 ± 83.5 | 0.952 (0.918–0.972) |
| ESV (mL) | 114.5 ± 78.5 | 0.955 (0.927–0.974) |
| LVEF (%) | 38.5 ± 13.6 | 0.932 (0.861–0.959) |
Interobserver Variability
The ICCs for the derived clinical parameters (EDV, ESV, and LVEF) were very high (>0.9) for the initial tracings ( Table 1 ). This shows excellent agreement among operators. The average differences and the percentage differences among operators are shown in Table 2 . The average difference for initial tracings for EDV was −3.3 ± 27.0 mL, for ESV was −2.7 ± 23.9 mL, and for LVEF was 0.5 ± 4.9 percentage points. The percentage differences for EDV, ESV, and LVEF were 10.4 ± 7.9%, 12.9 ± 10.6%, and 10.4 ± 9.1%, respectively. The mean values of mean absolute distances and Hausdorff distances are presented in Table 3 .
| EDV (mL) | ESV (mL) | LVEF (%) | ||||
|---|---|---|---|---|---|---|
| Mean ± SD | Percentage difference (%) | Mean ± SD | Percentage difference (%) | Mean ± SD | Percentage difference (%) | |
| All ( N = 45) | ||||||
| Initial tracings | −3.3 ± 27.0 | 10.4 ± 7.9 | −2.7 ± 23.9 | 12.9 ± 10.6 | 0.5 ± 4.9 | 10.4 ± 9.1 |
| After consensus | 1.5 ± 13.9 | 5.9 ± 4.5 | 1.1 ± 9.5 | 5.8 ± 4.9 | 0.1 ± 3.2 | 6.8 ± 5.7 |
| Good/fair image quality ( n = 30) | ||||||
| Initial tracings | −1.2 ± 17.7 | 9.6 ± 6.6 ∗ | −1.6 ± 17.2 | 8.4 ± 10.8 | 0.9 ± 4.9 | 10.5 ± 8.3 |
| After consensus | 0.8 ± 10.5 | 5.5 ± 3.5 ∗ | 0.4 ± 8.6 | 3.3 ± 4.0 | 0.5 ± 3.1 | 7.2 ± 6.0 |
| Poor image quality ( n = 15) | ||||||
| Initial tracings | −6.0 ± 35.6 | 13.2 ± 8.9 ∗ | −4.7 ± 33.4 | 8.3 ± 10.1 | −0.3 ± 4.9 | 10.2 ± 10.6 |
| After consensus | 1.7 ± 17.2 | 7.7 ± 5.9 ∗ | 2.5 ± 11.0 | 4.7 ± 6.1 | −0.6 ± 3.2 | 6.1 ± 5.0 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
