Background
Appropriate patient selection for transcatheter pulmonary valve (TPV) replacement requires accurate evaluation of right ventricular (RV) performance. The aim of this study was to evaluate the reliability and accuracy of echocardiography for evaluating RV parameters in patients in the five-center Melody TPV trial.
Methods
Echocardiographic data were compared with cardiac magnetic resonance (CMR) and catheterization; interobserver comparisons were made using site and core laboratory data.
Results
Doppler echocardiographic assessments of RV outflow tract obstruction and RV pressure showed excellent interobserver agreement; mean Doppler gradients were correlated most closely with gradients at catheterization ( R = 0.66), and Doppler RV pressure estimates were correlated well with catheterization data ( R = 0.58). Assessment of pulmonary regurgitation (PR) using a three-point severity scale showed good agreement with CMR-derived PR fraction (86%). The tricuspid annular Z score was highly reproducible but correlated weakly with CMR RV end-diastolic volume ( R = 0.21). However, RV apical diastolic area was highly reproducible ( R = 0.87) and had an excellent correlation with CMR RV end-diastolic volume ( R = 0.78); all patients with indexed RV apical diastolic areas ≥30 cm 2 /m 2 had CMR RV end-diastolic volumes ≥160 mL/m 2 . RV function using the fractional area change method showed a fair correlation with CMR RV ejection fraction ( R = 0.48).
Conclusions
In patients with dysfunctional RV outflow tract conduits, echocardiography provided reproducible, accurate estimates of pressure overload and RV size. Echocardiographic assessment of PR correlated less closely with CMR PR fraction but showed good categorical agreement; assessment of RV function by these methods was suboptimal. Echocardiography alone may be a suitable screening test for some TPV replacement candidates; CMR may be indicated for TPV replacement decisions hinging on assessment of RV function.
The availability of transcatheter pulmonary valve (TPV) technology is expanding. The appropriate proliferation of this technology will depend not only on the effectiveness and durability of the valves but on the ability to determine who stands to benefit from a transcatheter valve and how the relative risks, costs, and benefits of different treatment options balance out. The selection of patients for TPV implantation and assessment of outcomes will depend in part on how patients are evaluated, which imaging and physiologic testing modalities are used, and the ability to translate data between modalities.
The Melody TPV (Medtronic, Inc., Minneapolis, MN) was recently approved by the US Food and Drug Administration for treatment of patients with dysfunctional right ventricular (RV) outflow tract (RVOT) conduits. The hemodynamic entry criteria for the US Melody valve investigational device exemption (IDE) trial (Implantation of the Medtronic Melody Transcatheter Pulmonary Valve in Patients With Dysfunctional RVOT Conduits: A Feasibility Study) were based solely on echocardiographic evaluation, drawn largely from expert opinion, limited data, and logistic considerations. The present study was undertaken to evaluate the intertest accuracy of data obtained clinically using echocardiography, cardiac magnetic resonance (CMR), and invasive hemodynamic evaluation for assessment of right-heart parameters in patients undergoing catheterization for Melody valve placement as part of the US IDE trial. We hypothesized that echocardiographic measurements would correlate well with hemodynamic right-heart parameters at catheterization but poorly with CMR measures of RV size and function. Additionally, we sought to evaluate the interobserver variability between study site and core laboratory assessment of echocardiographic and CMR data to further characterize the accuracy and reproducibility of echocardiographic right-heart measurements in this patient population, with the hypothesis that echocardiographic measures of RV size and function would show less reproducibility than CMR measures. The point of these evaluations was to determine whether echocardiography performed in ambulatory clinical settings (i.e., in awake outpatients before catheterization) adequately defines right-heart hemodynamic and anatomic parameters compared with “gold-standard” CMR and catheterization measurements and thus whether echocardiography is suitable as a primary mode of evaluation before and after TPV implantation.
Methods
Study Design
The Melody valve is a catheter-delivered valve composed of two components: a balloon-expandable platinum-iridium stent and a segment of bovine jugular vein that contains a native venous valve. The valved segment of bovine jugular vein is sewn within the stent, with a continuous suture at each end of the stent and an interrupted suture attaching the vein wall to the stent at every node. A complete description of the five-center Melody valve IDE trial, including complete inclusion and exclusion criteria, was published previously. Trial inclusion criteria were age ≥5 years, weight ≥30 kg, existence of a circumferential RVOT conduit ≥16 mm in diameter at original implantation, and RVOT dysfunction defined according to the following echocardiographic criteria (site readings). For patients in New York Heart Association (NYHA) class I, study entry required an RVOT Doppler mean gradient ≥40 mm Hg and/or severe (4+) pulmonary regurgitation (PR) with RV dilation (defined as tricuspid valve [TV] annular dimension Z score ≥2.0, the maximal diameter as measured from apical four-chamber view during peak filling in early diastole) or RV dysfunction (defined as RV fractional area change <40%). For patients in NYHA class II or higher, study entry required an RVOT Doppler mean gradient ≥35 mm Hg and/or moderate PR (3+). Thus, depending on NYHA class, echocardiographic criteria on which study eligibility was based may have included mean RVOT gradient, PR severity, RV dilation (TV Z score), and RV function (RV fractional area change).
Precatheterization and follow-up testing were performed at predetermined intervals. This study includes data from precatheterization, early after implantation, at 6 months, and some 12-month follow-up studies. Results of echocardiographic and CMR studies were interpreted by specified investigators at each of the five study sites, as well as independent core laboratories. Angiographic images and catheterization-derived hemodynamic data were interpreted and recorded by site investigators only; there was no core laboratory for these variables. Primary outcome measures for the trial included predetermined definitions of safety, procedural success, and short-term effectiveness, as previously reported.
Echocardiography
Echocardiography was performed at all preimplantation and postimplantation time points. Patient-specific data obtained at each echocardiographic study included height, weight, heart rate, dominant rhythm, and systolic blood pressure. Standard echocardiographic views were used to obtain two-dimensional, spectral Doppler, and color Doppler recordings of all heart valves and chambers, with particular attention to the right-heart structures. All recorded Doppler velocities were the average of measurements from at least three cardiac cycles for patients in sinus rhythm and at least five cycles for patients not in sinus rhythm. The lateral dimension of the TV annulus was measured at end-diastole from the apical four-chamber view at the base of the leaflet attachments, using the image that provided the greatest dimension. If tricuspid regurgitation (TR) was present, continuous-wave Doppler evaluation was used to obtain the maximum regurgitant velocities from any transducer position. RV pressure was estimated from the TR jet velocity, but Doppler-predicted RV pressure was not adjusted for presumed or measured right atrial pressure.
PR was assessed from the parasternal short-axis view at the level of the great arteries, with evaluation of the proximal conduit as well as the main and branch pulmonary arteries (PAs; see Table 1 ). The grading of PR severity was based on synthesis of the color and pulsed Doppler evaluation of the width of the PR jet at the level of the proximal conduit or Melody valve and the degree of flow reversal in the main and branch PAs. Continuous-wave Doppler evaluation of the RVOT was used to assess maximum velocity and mean gradient across the conduit or Melody valve from parasternal long-axis views by trace integration of the modal velocities from spectral Doppler tracings, uncorrected for proximal velocity; for patients with inadequate parasternal views, apical or subcostal transducer tracings were used. PR was graded as none, trace (trivial), mild, moderate, or severe. These grades were also collapsed into a three-grade scale: none to mild, moderate, and severe.
Grade | Description |
---|---|
Color jet width PR | |
None | No diastolic color flow jet |
Trace | Pinhole color flow jet on ventricular side of leaflets |
Mild | Jet width <20% of the valve/conduit annular width |
Moderate | Jet width 20%–40% of the valve/conduit annular width |
Severe | Jet width >40% of the valve/conduit annular width |
Flow reversal criteria PR | |
None | No diastolic color flow reversal at level of valve/conduit |
Trace | Diastolic color flow reversal beginning at valve/conduit |
Mild | Diastolic color flow reversal extending above valve but confined to proximal half of the main PA |
Moderate | Diastolic color flow reversal extending into distal main PA |
Severe | Diastolic flow reversal extending into proximal branch PAs |
RV function was assessed quantitatively using the fractional area change method; area tracings were obtained from apical four-chamber views of the right ventricle at end-diastole and end-systole. Tracings were made along the endocardial border at the blood-tissue interface; RV fractional area change was defined using the formula ([end-diastolic area − end-systolic area]/end-diastolic area × 100). End-diastole was defined as the frame showing initial coaptation of the mitral leaflets, or the frame at which the QRS complex first occurred; end-systole was defined as the frame immediately preceding early diastolic mitral opening, or the frame with the smallest ventricular dimension if the mitral valve was not seen. RV and left ventricular internal diameters in systole and diastole were measured from two-dimensional parasternal short-axis images at the level of the papillary muscles.
CMR
CMR was performed before catheterization in patients meeting echocardiographic eligibility criteria and at 6 months after implantation, unless the patient had a pacemaker or other contraindication to CMR, but was not used to determine eligibility for the IDE trial. Standard CMR steady-state free precession sequences were used to obtain short-axis cine images covering both ventricles from apex to base. Ventricular mass (in grams) and volumes (in milliliters per square meter) were calculated by manually tracing the epicardial and endocardial contours of both right and left ventricles throughout the cardiac cycle; a series of parallel short-axis image slices were traced through the entire length of the ventricles, including the RV infundibulum up to the level of the conduit or pulmonary valve. To allow for measurement of antegrade and retrograde flow and derivation of PR fraction, flow measurements were obtained by performing a velocity-encoded phase contrast acquisition through an orthogonal plane at the level of the proximal main PA.
Cardiac Catheterization
Catheterization was typically performed under general endotracheal anesthesia. Standard right-heart and left-heart hemodynamic evaluation was performed, with subsequent angiography of the right ventricle, conduit, and aortic root. The peak-to-peak RV-to-PA gradient (in millimeters of mercury) was obtained from the difference between the systolic pressure of the RV body and the main PA distal to the conduit. Thus, this measurement included any subconduit obstruction but not branch PA obstruction. The peak RV pressure was converted for comparison with Doppler pressure estimation by subtracting the mean right atrial pressure (A-wave and V-wave pressures were not available for all the studies). Melody valve implantation was performed as previously described ; after implantation, hemodynamics and PA angiography were repeated.
Site and Core Laboratory Review
Raw data from all echocardiograms and CMR studies were forwarded to core labs, which repeated all required measurements and entered them into the same Web-based data collection system. Thus, for each study, both site and core lab data were recorded. There was no communication between site investigators and core lab readers, and data entered into the database from site readings were not available to core lab readers. Both site and core lab data entered into the system by the time of database closure (January 1, 2009) were used for this report. Because of the inevitable delay between follow-up evaluation and core lab reading, on-site measurements were usually entered into the data collection system before core readings. Thus, when the database was closed to generate the data set used for this analysis, more site data were available than core data. Only paired data (e.g., site and core, two different evaluation measures or modes) were used. Missing data were excluded on a case-by-case basis. The number of comparisons varied from 114 to 334, depending on the variable and type of comparison (e.g., site vs core, echocardiography vs magnetic resonance imaging [MRI]). The wide range of comparisons was due to a combination of factors, including variability in the number of relevant studies (echocardiographic studies were performed before implantation, after implantation, and at 6 and 12 months, whereas MRI was performed only before implantation and at 6 months), more or less complete patient data (e.g., MRI data were generally available in fewer patients than echocardiographic data, because of contraindications to MRI in some patients and ferromagnetic artifacts precluding certain measurements in others), and variable completion of core lab measurements. Only variables with paired measurements for a given patient were included, either paired core-site data or paired echocardiography-MRI or echocardiography-catheterization data.
Statistical Analysis
Preimplantation, early postimplantation, and 6-month and 12-month follow-up data were included if paired data were available (i.e., both site and core readings for interobserver comparisons and both echocardiography and the corresponding CMR or catheterization variable for intertest comparisons), without distinction between or adjustment for time points of the study (i.e., before catheterization, different postimplantation follow-up durations). Analysis of echocardiographic and CMR core laboratory versus site measurements was performed both by correlation parameter estimates and Bland-Altman analysis. For correlation parameter estimates of core laboratory versus site measurements, the core measurement was set as the y variable and the site measurement as the x variable, such that linear regression equations take the form core = (slope × site) + constant. For Bland-Altman analysis, the core laboratory value was used as the minuend in the difference calculation, and the bias was derived as bias = average of core measurement − site measurement, with 95% confidence intervals for the mean based on standard errors. Similar techniques were used to compare measurements between modalities (e.g., echocardiographically estimated and directly measured catheterization RV pressures). For correlation parameter estimates, measurement 1 (reference measurement) was set as the y variable and measurement 2 as the x variable. For Bland-Altman analysis, measurement 1 was used as the reference. Comparisons between site and core estimates of PR severity (ordinal grading) were performed using an unweighted κ test (presented with standard errors); an unweighted κ test was used because differences of more than one grade were rare, and we did not want to minimize the importance of one-grade differences. Comparison between the continuous PR fraction measured by CMR and the ordinal echocardiographic PR grade was performed with the Kruskal-Wallis test. CMR-derived PR fraction was then collapsed into ordinal grades empirically in an attempt to define a scale that corresponded to the three-grade echocardiographic scale: <15% = none to mild, 15% to 30% = moderate, and >30% = severe. For comparisons between modes of evaluation, both site and core readings were used without adjustment and measurements in the same patient at different time points were assumed to be independent. Study site was not considered as a modifying or confounding factor.
The study was conducted under an IDE (#G050186), and all versions of and amendments to the protocol were approved by the Food and Drug Administration, Center for Devices and Radiological Health, as well as the institutional review board at each institution. The trial is registered at ClinicalTrials.gov (identifier NCT00740870 ). Written informed consent was obtained before study enrollment.
Results
Patients
From January 2007 through November 2008, 99 patients were enrolled in the Melody IDE trial and underwent cardiac catheterization at a median age of 18 years (range, 7–44 years) and median weight of 60 kg (range, 27–118 kg). There were 63 male and 36 female patients. Primary diagnoses included tetralogy of Fallot (with or without pulmonary atresia) in 52 patients, conduit dysfunction after a Ross procedure in 17, transposition of the great arteries in 10, truncus arteriosus in eight, and other diagnoses in 12. In nine of these patients, Melody valves were not implanted, as reported previously. Precatheterization data for these patients were included in the present study. For paired echocardiographic and CMR studies, the median time between studies was 0 days (interquartile range, 0–2 days; range, 0–85 days). For paired echocardiographic and catheterization studies, the median time between studies was 1 day (interquartile range, 1–5 days; range, 0–56 days).
Interobserver Variability: Echocardiography
Comparisons of echocardiographic measurements between individual study sites and core laboratories are shown in Table 2 and Figure 1 . There was excellent agreement between site and core laboratories for Doppler echocardiographic measurements, including maximum and mean RVOT gradient and RV systolic pressure by TR jet, with minimal bias by Bland-Altman analysis, although variation was slightly increased with larger than average values. There was also good agreement for measurement of the TV annular diameter Z score and for echocardiographic measures of RV size, including RV short-axis internal diastolic dimension and RV apical systolic and diastolic areas. Despite the close relationship between site and core area measurements, there was moderate variability between site and core values for RV fractional area change.
Measurement | Median (minimum to maximum) value of all measurements | Intraclass correlation coefficient | Correlation coefficient ( R ) | Slope ∗ | Constant ∗ | Bland-Altman bias † (95% confidence interval) |
---|---|---|---|---|---|---|
Echocardiographic measurements | ||||||
Mean RVOT gradient (mm Hg) | 20.9 (4.0 to 71.3) | 0.93 | 0.94 | 0.970 | 2.360 | 1.70 (1.24 to 2.17) |
Maximum instantaneous RVOT gradient (mm Hg) | 31.4 (9.0 to 88.4) | 0.95 | 0.95 | 0.916 | 2.387 | −0.46 (−1.31 to 0.39) |
RV systolic pressure (mm Hg) | 61.8 (29.4 to 110.0) | 0.95 | 0.95 | 0.982 | 1.757 | 0.66 (0.05 to 1.28) |
RV diastolic area (cm 2 ) | 27.5 (9.0 to 79.2) | 0.78 | 0.87 | 0.865 | 8.406 | 4.78 (3.91 to 5.64) |
RV systolic area (cm 2 ) | 15.5 (4.0 to 74.8) | 0.81 | 0.87 | 0.956 | 4.146 | 3.44 (2.75 to 4.13) |
RV two-dimensional fractional area change | 39.9 (5.0 to 94.2) | 0.51 | 0.53 | 0.430 | 23.16 | −0.69 (−2.37 to 0.99) |
TV annular diameter Z score | 0.9 (−2.6 to 8.0) | 0.78 | 0.79 | 0.752 | 0.519 | 0.24 (0.08 to 0.40) |
RV internal diameter (cm) | 3.2 (1.4 to 7.2) | 0.75 | 0.85 | 0.853 | 0.937 | 0.50 (0.41 to 0.58) |
CMR measurements | ||||||
RV end-diastolic volume index (mL/m 2 ) | 108.3 (48.0 to 371.9) | 0.94 | 0.94 | 0.902 | 8.635 | −3.14 (−6.35 to 0.07) |
RV end-systolic volume index (mL/m 2 ) | 57.8 (11.5 to 331.1) | 0.90 | 0.91 | 0.954 | 9.203 | 6.16 (2.97 to 9.39) |
RV ejection fraction (%) | 45.7 (9.2 to 90.5) | 0.59 | 0.65 | 0.722 | 7.127 | −6.61 (−8.48 to −4.74) |
PR fraction | 9.4 (0.0 to 80.0) | 0.91 | 0.91 | 0.823 | 2.710 | −0.75 (−2.29 to 0.79) |
∗ For correlation parameter estimates, the core measurement is set as the y variable and the site measurement as x , such that linear regression equations take the form core = (slope × site) + constant.
† For Bland-Altman analysis, the core lab value was used as the minuend in the difference calculation: bias = average of core measurement − site measurement.
Echocardiographic assessment of PR, using the full grading scale outlined in the study protocol (see Table 1 , Figure 2 A), showed a modest correlation between site and core laboratories, with an unadjusted κ statistic of 0.426 (standard error, 0.035) and 59% agreement between site and core readings. When the “lesser” PR grades of none, trace, and mild were combined into a single ordinal category (none to mild), the unweighted κ statistic improved to 0.779 (standard error, 0.036), and site-core agreement improved to 90% ( Figure 2 B).
Interobserver Variability: CMR
The CMR-derived RV end-diastolic volume index showed excellent correlation between site and core laboratories, with minimal bias by Bland-Altman analysis ( Table 2 , Figure 3 ). RV end-systolic volume index also showed good agreement between the site and core assessments. RV ejection fraction was correlated moderately well; there was a systematic bias toward higher values by the sites than the core laboratory. Core and site estimates of PR fraction were correlated very well; there was a proportional bias only at higher than average PR values, with a trend toward higher PR fraction at sites than core laboratory. When PR fraction was collapsed into three ordinal categories as defined in the “Methods” section, there was 90% agreement between site and core CMR measurements.
Intertest Comparisons: Echocardiographic Correlation with Catheterization
Comparisons of selected right-heart variables across imaging modalities are listed in Table 3 . Cardiac catheterization was considered to be the gold-standard measurement for hemodynamic variables ( Figure 4 ). Echocardiographic estimates of RV pressure by TR velocity were correlated well with catheterization values corrected for mean right atrial pressure (RV systolic pressure − right atrial mean pressure; R = 0.61). In general, there was systematic overestimation of RV pressure by echocardiography. Doppler maximum RVOT gradients were correlated well with catheterization peak-to-peak RVOT gradient ( R = 0.54), with a trend toward gradient overestimation by Doppler echocardiography that was most pronounced at higher than average RVOT gradients. Mean Doppler RVOT gradients were correlated more closely ( R = 0.66) with catheter-measured values than were maximum Doppler RVOT gradients, with less bias by Bland-Altman analysis.
Measurement | Correlation coefficient ( R ) | Slope ∗ | Constant ∗ | Bland-Altman bias † (95% confidence interval) |
---|---|---|---|---|
1. Cath RV pressure − mean RA pressure (mm Hg) | 0.61 | 0.651 | 9.333 | −10.10 |
2. Echo RV pressure (mm Hg) | (−8.55 to −11.64) | |||
1. Cath peak-to-peak RVOT gradient (mm Hg) | 0.66 | 0.836 | 3.104 | −1.06 |
2. Echo mean RVOT gradient (mm Hg) | (−2.31 to 0.20) | |||
1. Cath peak-to-peak RVOT gradient (mm Hg) | 0.54 | 0.396 | 12.983 | −9.0 |
2. Echo maximum RVOT gradient (mm Hg) | (−21.7 to −16.4) | |||
1. CMR RV ejection fraction (%) | 0.48 | 0.579 | 22.98 | NA |
2. Echo RV 2D fractional area change (%) | ||||
1. CMR RV end-diastolic volume index (mL/m 2 ) | 0.24 | 7.563 | 112.175 | NA |
2. Echo TV annular diameter Z score | ||||
1. CMR RV end-diastolic volume (mL) | 0.84 | 7.753 | −24.396 | NA |
2. Echo RV diastolic area (cm 2 ) | ||||
1. CMR RV end-diastolic volume index (mL/m 2 ) | 0.78 | 6.960 | −1.516 | NA |
2. Echo RV diastolic area index (cm2/m 2 ) | ||||
1. CMR RV end-diastolic volume (mL) | 0.70 | 76.136 | −44.877 | NA |
2. Echo RV internal diameter (cm) | ||||
1. CMR RV end-systolic volume (mL) | 0.87 | 8.027 | −30.138 | NA |
2. Echo RV systolic area (cm 2 ) | ||||
1. CMR RV end-systolic volume index (mL/m 2 ) | 0.83 | 7.577 | −13.732 | NA |
2. Echo RV systolic area index (cm2/m 2 ) | ||||
1. CMR RV ejection fraction (%) | 0.59 | −1.434 | 61.646 | NA |
2. Echo RV systolic area index (cm2/m 2 ) |