Background
Multicenter clinical trials use echocardiographic core laboratories to ensure expertise and consistency in the assessment of imaging eligibility criteria, as well as safety and efficacy end points. The aim of this study was to report the real-world implementation of guidelines for best practices in echocardiographic core laboratories, including their feasibility and quality results, in a large, international multicenter trial.
Methods
Processes and procedures were developed to optimize the acquisition and analysis of echocardiograms for the Placement of Aortic Transcatheter Valves (PARTNER) I trial of percutaneous aortic valve replacement for aortic stenosis. Comparison of baseline findings in the operative and nonoperative cohorts and reproducibility analyses were performed.
Results
Echocardiography was performed in 1,055 patients (mean age, 83 years; 54% men) The average peak and mean aortic valve gradients were 73 ± 24 and 43 ± 15 mm Hg, and the average aortic valve area was 0.64 ± 0.20 cm 2 . The average ejection fraction was 52 ± 13% by visual estimation and 53 ± 14% by biplane planimetry. The mean left ventricular mass index was 151 ± 42 g/m 2 . The inoperable cohort had lower left ventricular mass and mass indexes and tended to have more severe mitral regurgitation. Core lab reproducibility was excellent, with intraclass correlation coefficients ranging from 0.92 to 0.99 and κ statistics from 0.58 to 0.85 for key variables. The image acquisition quality improvement process brought measurability to >85%, which was maintained for the duration of the study.
Conclusions
This real-world echocardiographic core lab experience in the PARTNER I trial demonstrates that a high standard of measurability and reproducibility can result from extensive quality assurance efforts in both image acquisition and analysis. These results and the echocardiographic data reported here provide a reference for future studies of aortic stenosis patients and should encourage the wider use of echocardiography in clinical research.
Large multicenter cardiovascular clinical trials benefit greatly by the use of echocardiographic core laboratories (ECL) to ensure expertise and consistency in the assessment of imaging eligibility criteria, as well as safety and efficacy end points. Complete and reproducible imaging in a large clinical trial requires a substantial effort related to study design and core laboratory setup, establishment of infrastructure, site and analysis lab quality assurance, and other procedures and policies designed to optimize results. Although societal guidelines for best practices in ECLs have been published, there are few reports of their implementation.
The Placement of Aortic Transcatheter Valves (PARTNER) I trial is a large, two-arm, international randomized trial of patients with severe aortic stenosis (AS) that illustrates the use of an ECL to provide critically important, unbiased, reproducible echocardiographic assessments. Within the trial, cohort A compared transcatheter aortic valve (AV) replacement (TAVR) with surgical AV replacement (SAVR) in high-risk, operable patients; TAVR was noninferior. Cohort B compared TAVR with medical therapy in patients with inoperable AS; TAVR was superior. As the largest prospective randomized study of a single valve lesion ever performed, the PARTNER I trial is an ideal setting in which to examine core laboratory methodology and quality results.
In this report, we present (1) the feasibility, quality, and real-world implementation of ECL methodology; (2) echocardiographic descriptions of a large contemporary population with severe AS in the PARTNER I trial; and (3) recommendations for future efforts using ECLs in multicenter clinical trials.
Methods
Patient Population and Echocardiographic Assessments
The PARTNER I trial enrolled patients with severe AS, defined as AV area (AVA) < 0.8 cm 2 and either a mean AV gradient ≥ 40 mm Hg or a peak aortic jet velocity ≥ 4.0 m/s, as assessed by sites. Patients were divided into two cohorts. Patients in cohort A were candidates for surgery, but at high 30-day risk (Society of Thoracic Surgeons risk score ≥ 10% or other coexisting conditions predicting ≥15% mortality). Those in cohort B were not candidates for surgery, because of coexisting conditions predicting ≥50% mortality or serious irreversible conditions. Pertinent inclusion and exclusion criteria have been previously described, as have the clinical characteristics of the enrolled populations. Between May 2007 and August 2009, 1,057 patients were enrolled in the trial at 26 sites, 699 in cohort A and 358 in cohort B. Patients underwent transthoracic echocardiography at baseline and at approximately 7 days, 30 days, 6 months, and 1, 2, 3, 4, and 5 years after randomization.
ECL Structure and Processes
The PARTNER I ECL at the Duke Clinical Research Institute includes lab medical and technical directors, a principal investigator (P.S.D.), a project leader, a lead technician (G.D.), a clinical trial coordinator (L.D.), data management specialists, statisticians, and information technology experts as well as sonographer and physician readers. All personnel were Collaborative Institutional Training Initiative certified and followed good clinical practice and analysis laboratory standard operating procedures as delineated in the 2009 American Society of Echocardiography (ASE) best practices document and US Food and Drug Administration Code of Federal Regulations Part 11 requirements and draft guidance on heart valve assessments and imaging in clinical trials. All hardware and software were prospectively validated, and an internal audit ensured regulatory compliance. The ECL analysis team consisted of two experienced physician echocardiographers (each with level 3 training and >25 years of experience) and 15 sonographers (each with registry credentials and >3 years of experience).
ECL Study Work Flow
Study setup included creation of the image acquisition protocol and site manual, image analysis protocol, and programming the analysis and electronic data capture systems (InForm; Oracle Health Sciences, Redshores, CA). ECL staff members underwent a comprehensive orientation, covering the PARTNER I trial goals and clinical protocol, the image acquisition protocol and site manual, the imaging case report form, the image analysis protocol, site and core laboratory quality procedures, standardized measurements, and data verification checks and standards.
Reproducibility Testing
All sonographers and physicians at the ECL were required to meet laboratory standards for intrareader and interreader reliability and were tested on eight variables: biplane left ventricular (LV) ejection fraction (LVEF), visual LVEF, peak and mean AV gradients, AVA, and the severity of paravalvular and transvalvular aortic regurgitation (AR) and mitral regurgitation (MR). The 30 echocardiograms used for testing were selected randomly from among the early images submitted for the trial that had measureable data for all variables of interest. For each intrareader and interreader reliability variable, the maximal acceptable variation within and between readers was prospectively determined as ≤10% for biplane LVEF, ≤10% for visual LVEF, ≤10 mm Hg for peak and mean AV gradients, ≤0.3 cm 2 for AVA, and ≤1 grade for paravalvular and transvalvular AR and MR. Eighty percent of all possible pairwise comparisons from all readers were required to fall within these ranges and 100% of differences within twice these ranges. Readers falling short of these standards received additional training and coaching, reviewing sources of errors, and then repeated the testing on a different set of PARTNER I echocardiograms until reaching the same criteria for reproducibility. To enhance uniformity and reliability, group reading sessions were held quarterly to review protocols, sources of error, outliers, reader idiosyncrasies, variable or missing measurements, and reading instructions.
Image Acquisition Design and Site Quality Assurance
Each site underwent Web-based training by core laboratory staff members. Sites were required to submit one AS echocardiogram adequate for ECL analysis to qualify for enrollment; additional studies were requested until all requirements were met. Only two sites required submission of a second set of images for qualification. Site quality assurance activities continued throughout the course of the trial. Upon receipt in the core laboratory, every echocardiogram underwent administrative and technical review to ensure completeness and measurability of key parameters; written quality feedback was provided to site staff members. Site performance across all studies on the key variables noted above was tracked, and monthly and quarterly reports were issued to sites beginning in the fourth quarter of 2008, including anonymous benchmarking relative to other sites ( Figure 1 A). Sites with measurability < 85% completed intensive retraining, including in-depth Web conferences reviewing the acquisition protocol, feedback form, and benchmarked scorecard. When needed, one-on-one reviews of specific images that met, or did not meet, adequacy standards were conducted between site and core lab personnel.
Image Transfer and Management
Images were transmitted by compact disc, overnight courier, and a Web-based picture archiving and communication system (HeartIT; Heart Imaging Technologies, Durham, NC). On receipt at the ECL, images were transferred in digital format to an analysis workstation (Digiview; Digisonics Inc, Houston, TX) with backup storage on a secure structured query language server.
Study Interpretation
Echocardiographic measurements were performed by ECL sonographers and reviewed and amended by ECL physicians. Average time per study was 1.5 hours and 0.20 hours, respectively, but varied depending on image quality, complexity, and heart rhythm (more measurements were required in patients with atrial fibrillation). All measurements and analyses were performed without knowledge of clinical or other laboratory data, including prior echocardiographic results and group assignment.
Final Data Consistency and Quality Checks
Final echocardiographic measurements were uploaded into a custom database and were subjected to prespecified automated range checks that triggered queries for study review. All queries were reviewed by a sonographer and approved by a physician overreader. The data presented here are derived from a November 2011 extract. Additional measurements and assessments performed after valve replacement or insertion are available in Appendix 2 .
Echocardiographic Measurements
An average of three cardiac cycles was used for patients in sinus rhythm, and an average of five cardiac cycles was used for those in atrial fibrillation. All chamber parameters were measured according to the recommendations of the ASE.
LV Measurements
LV diameters and wall thicknesses were measured from the two-dimensional parasternal long-axis view of the left ventricle at the chordal level just below the mitral leaflet tips in end-systole and end-diastole. Relative wall thickness was measured according to recent literature as the mean of the interventricular septal dimension (IVSd) and LV posterior wall dimension (LVPWd) divided by LV end-diastolic (LVED) dimension and also as (2 × LVPWd)/LVED dimension. LV mass was calculated using the ASE-recommended formula and indexed to body surface area:
LV mass = 0.8 × { 1.04 [ ( LVED + LVPWd + IVSd ) 3 − ( LVED ) 3 ] } + 0.6 g .
LV volumes and LVEF were measured using the biplane Simpson’s volumetric method combining apical four-chamber and two-chamber views. The LV endocardial border was traced contiguously from one side of the mitral annulus to the other, excluding the papillary muscles and trabeculations, and any apical tethering of the mitral leaflets. In the very small number of images (<1%) with microbubble contrast, borders were traced similarly. LVEF was also determined by visual estimation (in 5-point increments) and, when the definition of the LV endocardial border was not adequate for biplane tracing, was substituted to provide a single “combined LVEF” determination in all patients. Use of alternative methods involving single-plane or LV diameter measurements were avoided because of the high prevalence of coronary artery disease, resulting in irregularly shaped ventricles. Stroke volume, cardiac output, and cardiac index were calculated using the LV volumes measured with the biplane Simpson’s method. Stroke volume was also calculated by Doppler using the time-velocity integral of the LV outflow tract (LVOT) and LVOT diameter as
stroke volume = ( LVOT diameter × LVOT diameter × 0.785 ) × LVOT time – velocity integral .
Aortic annular and root measurements were obtained at end-diastole on a zoomed parasternal long-axis view or the most measurable image. The aortic annulus was measured from the hinge point of the anterior aortic cusp and the ventricular septum to the junction of the posterior aortic cusp and the anterior mitral leaflet. Additional end-diastolic measurements included the aortic root (maximal diameter at the sinuses of Valsalva), ascending aorta, and sinotubular junction. It is important to note that the trial used site-obtained midsystolic measurements for SAPIEN (Edwards Lifesciences, Irvine, CA) device sizing by both transthoracic and transesophageal echocardiography, as experience suggests that systolic measurements are both slightly larger and more useful for device sizing. LVOT diameter was measured in midsystole, no more than 0.5 cm apical to the annular measurement (see above), and visually avoiding the septal bulge, dystrophic calcification, or systolic anterior motion of the mitral leaflets within the LVOT ( Figure 2 ).
AV Assessment, Hemodynamics, and Regurgitation
The core laboratory followed the ASE and European Association of Echocardiography guideline for assessing the severity of native valvular stenosis and regurgitation. Qualitative AV assessments included leaflet thickening, calcification, and mobility, graded as none, mild, moderate, or severe. AV peak and mean gradients were obtained using the view showing the maximal velocity. AVA or effective orifice area was calculated according to the continuity equation and indexed to body surface area. Machine settings used to obtain the best view of AR (when present), including the Nyquist limit (centimeters per second), color gain (decibels), depth (centimeters), frame rate (Hertz), persistence, and smoothing were recorded and integrated with qualitative assessments of jet area and height. Recordings not made with the appropriate Nyquist limit (50–60 cm/sec) or adequate frame rate were discounted.
AR severity was assessed in all relevant views using color and spectral Doppler. Transvalvular regurgitation was graded according to ASE and European Association of Echocardiography recommendations, integrating all available and reliable parameters, as follows: none = no AR jet; trivial = trivial amount of regurgitation (jet width is <10% of LVOT); mild = small, central jet of regurgitation (jet width is 10%-24% of LVOT), vena contracta < 0.3 cm, incomplete or faint jet density by continuous-wave Doppler, and normal LV size (unless there are other reasons for LV dilation); moderate = small or moderate eccentric jet or moderate central jet of regurgitation (jet width is 25%–64% of LVOT), vena contracta > 0.3 cm but < 0.6 cm, and no signs of severe AR; and severe = moderate or large eccentric jet or large central jet of regurgitation (jet width is ≥65% of LVOT), vena contracta ≥ 0.6 cm or holodiastolic flow reversal in descending aorta, dense jet density by continuous-wave Doppler, and dilated left ventricle.
Mitral Valve Assessments
Mitral valve thickening was assessed as none, trace, mild, moderate, or severe, and MR severity was assessed according to ASE criteria as 0 to 4+, or none, trivial, mild, moderate, or severe.
Statistical Analyses
Descriptive statistics are presented to summarize continuous variables as mean ± SD, median (interquartile range), and minimum and maximum, and categorical variables are expressed as percentage and frequency counts, both by group and overall. Wilcoxon’s rank-sum test was conducted to compare the medians of continuous clinical variables between groups, while χ 2 or Fisher’s exact tests were used to compare the percentages of categorical variables between groups. P values < .05 were considered statistically significant.
Correlations were performed using Pearson’s correlation coefficients. Intraclass correlation coefficients and intraclass κ statistics were used to assess interobserver and intraobserver variability.
Results
Quality Assurance
Site Image Acquisition Quality
The mean initial site measurability score was 78%, and the median initial score was 85%. Overall, 14 sites (54%) achieved >85% measurability on initial assessment, with scores ranging from 17% to 100%. After the initiation of benchmarked scorecards in the fourth quarter of 2008, site measurability scores rose immediately and substantially, with only two of the 12 sites failing to show improvement. These sites were subsequently dropped from the study. During the course of the trial, the quarterly monitoring detected a drop in measurability scores to <85% at eight previously adequate sites (31%), with the most common reason being a change in personnel. After retraining as described above, six of these sites (75%) were able to improve measurability to >85% in the subsequent quarter ( Figure 1 B).
Core Lab Reproducibility
Reproducibility by the coverage probability method was determined on 649 to 1,360 pairwise comparisons between readers for each of the eight variables of interest on 30 echocardiograms (total number of comparisons, 8,031). Intraclass correlation coefficients subsequently calculated across all readers for continuous variables were quite high, ranging from 0.92 to 0.99 for physician overreaders and from 0.89 to 0.97 for sonographers. Similarly, the κ statistics for agreement for categorical variables calculated for physician readers were good (0.56–0.85; Tables 1 and 2 ).
| Reader type | Analysis | Biplane LVEF | Visual LVEF | Mean AV gradient | Peak AV gradient | AVA |
|---|---|---|---|---|---|---|
| Sonographer | Intraobserver | 0.70-0.87 | 0.95-0.99 | 0.99-1.00 | 1.00 | 0.91-1.00 |
| Interobserver | 0.89 | 0.88 | 0.97 | 0.97 | 0.90 | |
| Physician | Intraobserver | 0.56-0.94 | 0.98-0.99 | 0.99-1.00 | 0.99-1.00 | 0.95-0.97 |
| Interobserver | 0.92 | 0.95 | 0.99 | 0.99 | 0.95 | |
| Pairwise comparisons | 649 | 1,101 | 1,065 | 1,077 | 1,059 |
| Reader type | Analysis | Transvalvular AR | Paravalvular AR | MR |
|---|---|---|---|---|
| Sonographer | Intraobserver | 1.00 | 0.86-0.87 | 0.68-0.86 |
| Interobserver | 0.349 | 0.445 | 0.411 | |
| Physician | Intraobserver | 0.64-0.85 | 0.58-0.73 | 0.74-0.85 |
| Interobserver | 0.579 | 0.854 | 0.847 | |
| Pairwise comparisons | 1,360 | 1,360 | 1,360 |
Echocardiographic Results
Patient Population
Echocardiographic images were available for analysis by the core laboratory at baseline in 1,055 of 1,057 randomized patients. In those with baseline studies, the mean age was 83.8 ± 7.3 years, and 54% were men. AVA measurements were obtainable in 973 patients (92%) and biplane LVEFs in 592 (56%). Inability to measure a biplane LVEF was associated with higher body mass index, coronary artery bypass grafting history, and diabetes but not chronic obstructive pulmonary disease, renal failure, or hypertension ( Table 3 ).
| Variable | AVA at baseline ( n = 973) | Missing AVA at baseline ( n = 82) | P | Biplane LVEF at baseline ( n = 592) | Missing Biplane LVEF at baseline ( n = 463) | P ∗ |
|---|---|---|---|---|---|---|
| Age (y) | 83.81 ± 7.30 | 83.13 ± 7.21 | .32 | 84.34 ± 6.97 | 83.01 ± 7.63 | .010 |
| Men | 53.6% | 52.4% | .83 | 51.4% | 56.4% | .10 |
| BMI (kg/m 2 ) | 26.97 ± 6.70 | 27.40 ± 7.32 | .82 | 26.31 ± 5.92 | 27.89 ± 7.58 | .002 |
| Systolic blood pressure † (mm Hg) | 127 ± 22 | 127 ± 23 | .83 | 128 ± 22 | 127 ± 22 | .51 |
| Diastolic blood pressure † (mm Hg) | 66 ± 12 | 65 ± 12 | .41 | 66 ± 12 | 66 ± 12 | .73 |
| STS score | 11.69 ± 4.18 | 11.53 ± 3.91 | .82 | 11.72 ± 4.20 | 11.62 ± 4.11 | .78 |
| Cardiovascular risk factors | ||||||
| Any diabetes | 37.8% | 48.8% | .05 | 36.0% | 42.1% | .04 |
| Hyperlipidemia | 79.5% | 81.7% | .64 | 79.2% | 80.3% | .65 |
| Smoking | 48.4% | 42.7% | .32 | 48.3% | 47.5% | .80 |
| Hypertension | 89.5% | 90.2% | .83 | 88.8% | 90.5% | .38 |
| Cardiovascular conditions | ||||||
| Angina | 25.4% | 14.6% | .03 | 24.3% | 24.8% | .85 |
| CHF | 97.9% | 97.6% | .69 | 97.5% | 98.5% | .25 |
| CAD | 74.1% | 72.0% | .94 | 73.6% | 74.3% | .81 |
| Prior CABG | 24.7% | 50.0% | .04 | 37.0% | 45.8% | .004 |
| Cardiomyopathy | 18.9% | 15.9% | .50 | 18.0% | 19.5% | .53 |
| Pulmonary hypertension | 48.5% | 45.1% | .56 | 47.5% | 49.2% | .57 |
| Noncardiac conditions | ||||||
| Renal disease (creatinine ≥ 2 mg/dL) | 19.6% | 12.2% | .10 | 20.6% | 16.9% | .12 |
| COPD | 23.9% | 31.0% | .23 | 22.1% | 27.6% | .09 |
| Chest wall radiation | 2.4% | 2.4% | 1.0 | 2.2% | 2.6% | .69 |
| Chest wall deformities | 2.6% | 1.2% | .72 | 2.2% | 2.8% | .53 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
