The Ventricular Volume Variability Study of the Pediatric Heart Network: Study Design and Impact of Beat Averaging and Variable Type on the Reproducibility of Echocardiographic Measurements in Children with Chronic Dilated Cardiomyopathy




Background


Clinical trials often rely on echocardiographic measures of left ventricular size and function as surrogate end points. However, the quantitative impact of factors that affect the reproducibility of these measures is unknown. To address this issue, the National Heart, Lung, and Blood Institute–funded Pediatric Heart Network designed a longitudinal observational study of children with known or suspected dilated cardiomyopathy aged 0 to 22 years from eight pediatric clinical centers.


Methods


Clinical data were collected together with 150 echocardiographic indices of left ventricular size and function. Separate observers performed duplicate echocardiographic imaging. Multiple observers performed measurements from three cardiac cycles to enable assessment of intraobserver and interobserver variability. The impacts of beat averaging (BA), observer type (local vs core), and variable type (areas, calculations, dimensions, slopes, time intervals, and velocities) on measurement reproducibility were studied. The outcome measure was percentage error (100 × difference/mean).


Results


Of 173 enrolled subjects, 131 met criteria for dilated cardiomyopathy. BA, variable type and observer type all influenced percentage error ( P < .0001). Core interobserver percentage error (medians, 11.4%, 10.2%, and 9.3% for BA using one, two, and three beats, respectively) was approximately twice the intraobserver percentage error (medians, 6.3%, 4.9%, and 4.2% for BA using one, two, and three beats, respectively). Slopes and calculated variables exhibited high percentage error despite BA. Chamber dimensions, areas, velocities, and time intervals exhibited low percentage error.


Conclusions


This comprehensive evaluation of quantitative echocardiographic methods will provide a valuable resource for the design of future pediatric studies. BA and a single core lab observer improve the reproducibility of echocardiographic measurements in children with dilated cardiomyopathy. Certain measurements are highly reproducible, while others, despite BA, are poorly reproducible.


Left ventricular (LV) size and function are important independent predictors of outcomes in numerous forms of cardiovascular disease. In children, echocardiography is the primary modality used to assess ventricular function, and echocardiographically derived measurements are commonly used as end points in pediatric clinical trials. Although there is extensive experience with this technology, there are few quantitative data concerning reproducibility of these measurements, particularly in children with dilated cardiomyopathy (DCM). This issue is particularly problematic because of the wide range of factors that are known to affect this reproducibility. Patient age and habitus are known to be important, as is disease status. Less commonly appreciated is the evolution of technology over time, which requires that this issue be addressed anew with each new generation of echocardiographic equipment. Potential sources of variability in echocardiographic measurements include all of the following:



  • 1.

    interpatient variability: differences among patients related to patient-specific factors such as age, body size, variation in cardiac output, physical training, and so on, and disease-specific factors such as the severity of disease and treatment status;


  • 2.

    interstudy variability: longitudinal variation within an individual due to physiologic factors, variation in treatment, and change in disease status;


  • 3.

    intrastudy variability: short-term (beat-to-beat) variations in the same patient secondary to respiratory effects or change in position and minute-to-minute variation related to factors such as change in emotional state; and


  • 4.

    technical factors that modulate each of the foregoing, which include



    • a.

      variability due to differences in echocardiographic and image analysis equipment and whether sedation was used,


    • b.

      intraobserver and interobserver variability in data (image) acquisition, and


    • c.

      intraobserver and interobserver variability in measurements, including frame selection and structure identification.




In addition to facilitating efforts to improve the reproducibility of echocardiographic data, evaluation of variability serves to identify which measured or calculated variables may be preferable and which are too poorly reproducible to be clinically reliable. Finally, the design of clinical trials requires estimation of the number of patients that must be enrolled, which requires knowledge of total variability.


This report describes the design and initial findings of the Ventricular Volume Variability Study, a multicenter study in children with DCM. We present first the overall study aims, the complex study design implemented to address these aims, and the study results specific to the impact of beat averaging. There are few data that examine the impact of beat averaging and variable type on the interobserver and intraobserver reproducibility of echocardiographic measurements. We elected to perform these analyses before addressing the primary aim of the overall study, so that decisions regarding beat averaging could be applied to all subsequent study analyses. The Ventricular Volume Variability Study was conducted to address the following aims.


The primary aim was to determine the interstudy variability of echocardiographically derived LV end-diastolic volume Z score, mass Z score, and ejection fraction Z score in pediatric patients with DCM, more specifically the variance at a single point in time as well as the variance of change in measurements over time.


Secondary aims included the following:



  • 1.

    to determine the relative magnitude of the various sources of variability in echocardiographic outcomes to optimize operational procedures that can minimize variance,


  • 2.

    to determine the interstudy variability of echocardiographically derived indices of LV systolic and diastolic function, and


  • 3.

    to determine the relationship of clinical status, including treatment, to the interstudy variability and repeatability of echocardiographic measurements.



The specific analysis presented in this report had three aims: (1) to determine whether the choice of single-beat analysis or two-beat or three-beat averaging has an influence on interobserver and intraobserver reproducibility of echocardiographic measurements and calculated variables, (2) to examine the effect of the type of echocardiographic variable on reproducibility as it pertains to beat averaging, and (3) to evaluate the effect of beat averaging on the reproducibility of measurements that were performed at the local centers compared with those performed at the core laboratory.


Methods


Subjects


Pediatric patients with known or suspected DCM were enrolled at the time of echocardiographic presentation at each of the eight study centers (see “Acknowledgments”) between May 2005 and July 2007. Inclusion criteria were age < 22 years, known or suspected DCM, disease duration > 2 months, anticipated longitudinal follow-up to occur at the same institution, and the provision of informed consent. Exclusion criteria (listed in detail in Appendix Figure A1 ) included other forms of cardiomyopathy and congenital heart disease. Exclusion criteria also included noncompaction (due to an inability to reliably define LV endocardial borders), excessive nonsinus rhythm (due to excess beat-to-beat variance), and hemodynamic instability (due to the intent to assess longitudinal natural history).


Training


The principal investigator from each clinical site along with one or more designated site sonographers attended an in-person training session that included protocol review and demonstration of the image acquisition techniques. The measurement methods were reviewed in detail with the site principal investigator and the designated primary sonographer for the study, one of whom performed all of the local measurements. Each center performed and submitted three practice echocardiograms to the core lab for review and feedback. After approval, enrollment commenced.


Demographics


Height and weight were measured and body surface area was calculated using the Haycock formula. Systolic, diastolic, and mean blood pressures were recorded four times with the patient in a recumbent position using an automated blood pressure device. The values from the first recording were discarded, and the average value of the other three measurements was calculated for each pressure.


Echocardiographic Acquisition


All echocardiograms were obtained according to a standardized protocol, with acquisition of the images listed in Appendix Table A1 . Each center designated one or more “primary” sonographers who participated in hands-on study training sessions. For each echocardiographic evaluation, image acquisition was performed by a “primary” echocardiographer, and then a second image acquisition was independently performed by any available experienced sonographer (who had not necessarily participated in the study training sessions). The second set of images was acquired immediately after the first.


Eligibility for Longitudinal Evaluation


A standardized set of measurements was performed at the study center on the primary data acquisition, and the results of these measurements were used to determine eligibility for longitudinal echocardiographic evaluation. Patients who met criteria for the diagnosis of DCM on the basis of the inclusion and exclusion criteria listed above who also were found to have both LV dilation (defined as end-diastolic dimension > 5.5 cm or end-diastolic dimension Z score > 2) and LV dysfunction (defined as ejection fraction < 50%, shortening fraction < 28%, or Z score for either parameter < −2) were judged eligible for longitudinal assessment ( Appendix Figure A2 ). Eligible patients underwent repeat echocardiographic assessments according to the study protocol on return visits up to 18 months after enrollment, providing no indication for withdrawal from the study was met. Indications for withdrawal were death, cardiac transplantation or LV reduction surgery, institution of an LV assist device including extracorporeal membrane oxygenator support, and patient or physician preference. Each subject, therefore, was expected to have two sets of echocardiographic images at the time of study enrollment and two sets from at least one follow-up visit. The target timing for the follow-up echocardiographic assessment was 12 months, because this was considered to be a likely interval to assess change in a randomized trial. In the completed study, the mean time between paired echocardiographic assessments using the follow-up echocardiogram closest to the 12-month target was 9.1 ± 3.5 months (median, 9.6 months; range, 2–18 months).


Partial Participation Group


Subjects who underwent baseline image acquisition but were then found to not meet criteria for longitudinal evaluation were excluded from further participation, but their baseline images were included in the analyses of reproducibility, thereby permitting the evaluation of reproducibility over a broader range of ventricular size and function in addition to enhancing the analysis of the impact of severity of dysfunction on reproducibility.


Echocardiographic Analysis


A standardized measurement protocol that included a total of 150 measurements and calculated variables was performed at the core laboratory for each of three cardiac cycles (450 measurements total) for each set of echocardiographic images ( Appendix Table A2 ). Measurements were categorized as areas (nine variables); calculated variables derived from two, three, and four measured variables (19, 25, and 21 variables, respectively); dimensions (16 variables); electrocardiographic time intervals and heart rates (seven and nine variables, respectively); integrals (one variable); slopes (four variables); Doppler and M-mode time intervals (22 variables); and Doppler velocities (17 variables). A single observer at each center performed the measurements for the primary image acquisition at the initial and follow-up visits using locally available technology and submitted the results to the data coordinating center. The data sets constructed at the local centers were composed of 119 of the above 150 measurements and derived indices. The images from both the primary and secondary image acquisitions from each subject visit were submitted to the data coordinating center, where they were blinded to study date and patient identification and coded before transmission to the echocardiographic core lab for analysis ( Appendix Figure A1 ). Image capture and transfer included videotape, analog-to-digital converted images, and Digital Imaging and Communications in Medicine, depending on the date and the center. All echocardiographic measurements were performed using custom Digital Imaging and Communications in Medicine software (EchoTrace; Marcus Laboratories, Boston, MA).


At the core lab, a primary and secondary core lab observer analyzed both the primary and secondary image acquisitions using the same measurement protocol ( Appendix Table A2 ). In addition, the primary core lab observer repeated the measurements on blinded sets of the primary images at 1 month and 1 year after the original measurements. Primary and secondary image sets acquired at follow-up evaluations of fully eligible subjects were each analyzed by both the primary and secondary core lab observers. Altogether, there were 12 different categories of echocardiographic data sets for statistical comparison, as listed in Appendix Table A3 .


Image Quality Assessment


In addition to performing the measurements, the primary core lab observer performed a quality assessment for each of the images in Appendix Table A1 . The image grading system was defined as follows:



  • 1.

    Excellent: The full extent of the structure boundary of interest was clearly defined, with no visible gaps; zoom mode was activated to maximize image resolution for two-dimensional structures; and Doppler signals were scaled proportional to image size.


  • 2.

    Good: The full extent of the structure boundary of interest was contained within the image sector, with only brief gaps requiring interpolation, and for Doppler signals, there was minimal baseline artifact.


  • 3.

    Fair: Nearly the entire boundary of the structure of interest was contained and adequately visible, but minor extrapolation beyond the imaging sector was required; boundaries had identifiable but indistinct borders; and prominent baseline artifact was present on Doppler recordings.


  • 4.

    Poor: Significant portions of the boundary of interest required interpolation from visible but indistinct segments, or more than a short segment of the boundary was outside of the imaging sector.


  • 5.

    Unusable: The structure was not recorded or was too poorly defined to be measured.



Additionally, each echocardiogram was also evaluated for (1) trabeculations that could potentially interfere with definition of the LV apical endocardium, (2) qualitative assessment of regional wall motion abnormalities, and (3) septal displacement resulting in a noncircular short-axis LV configuration.


Clinical Data Collection


The medical records of all subjects were reviewed at the time of enrollment, at each subsequent echocardiographic assessment, and at 18 months after enrollment for medical history, including medical therapy and changes in medical therapy, procedures, interventions, adverse events, and symptom and cardiac transplantation listing status.


Statistical Analysis


The primary aim of this study was to determine the interstudy variability of echocardiographically derived LV measurements in pediatric patients with DCM, in particular the variance of LV ejection fraction Z score at a single point in time and the variance of change in Z score between two time points. The required sample size to estimate the population standard deviation to within a prespecified tolerance is n = Z 1−σ/2 /(2 d 2 ), where d is the allowed fractional deviation from σ, the population sample standard deviation. To construct a two-sided 95% confidence interval for σ that deviates no more than 15% from the true value requires a sample size of 86. It was estimated that 25% to 30% of subjects would withdraw, undergo echocardiography under differing sedation conditions, or have incomplete baseline or follow-up echocardiogram pairs for other reasons. Therefore, the target sample size was set to be 120 patients with qualifying baseline echocardiograms to ensure that 86 of these had paired interpretable echocardiograms (a total of 172 echocardiograms) obtained under similar conditions. In actual execution, the study enrolled 173 patients, of whom 131 were eligible for longitudinal evaluation, to ensure that 86 qualifying echocardiogram pairs would be obtained. A total of 107 of these subjects had at least one follow-up echocardiogram submitted, and 97 of these formed qualifying echocardiogram pairs. The 10 subjects with nonqualifying pairs met one of the following criteria: no secondary image acquisition, incorrect sonographer, or inconsistent sedation status between the two echocardiograms.


Ventricular Volume Variability Study Analysis Plan


On the basis of the study design shown in Appendix Table A3 , for any given echocardiographic parameter, there were 21 or 15 sets of measurements from each study visit (3 sequential cardiac cycles × 7 different readings for a baseline visit and 5 different readings for a follow-up visit). The analyses of these data sets include the following comparisons of interest to assess:


Interacquisition Observer Variability


1. Images acquired by different observers and measured by the same observer.


Intraobserver Variability


2. The same set of images measured twice by the same person spaced by 1 month.


Intraobserver Drift


3. The same set of images measured twice by the same person spaced by 12 months.


Interobserver Variability


4. The same set of images, measurements by one core lab observer versus measurements by a second core lab observer.


5. The same set of images, measurements by core lab versus measurements by study center.


Changes over Time in Cardiac Function Holding Acquisition Observer and Measurer Constant


6. Images acquired by the same observer and measured by the same observer.


Statistical Methods: Beat Averaging


The outcome measure was the percentage error of the mean. Echocardiographic measurements using the primary image acquisition from all baseline studies were included in analysis. For each evaluation of reproducibility between two measurements of the same entity, the difference (“error”) between the two measurements was divided by the mean of those two measurements. Three settings were used to compare the following two measurements:




  • core laboratory interobserver variability: primary versus secondary observer;



  • core laboratory intraobserver variability: primary observer immediate versus 1-month reading; and



  • core versus local laboratory interobserver variability: primary core laboratory observer versus clinical center observer.



We fit a mixed-effects model with estimates obtained by restricted maximum likelihood with a compound symmetry covariance structure (fixed effect for beat-averaging method and random effect for subject) to assess whether percentage error significantly differed for single-beat versus three-beat average and two-beat versus three-beat average, for interobserver, intraobserver, and local versus core laboratory comparisons. We fit a mixed-effects model with unstructured covariance to assess whether, for each variable type, on the basis of three-beat-averaged measurements, interobserver percentage error differed for local/core versus core/core percentage error estimates and for core lab interobserver versus intraobserver percentage error.

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Jun 7, 2018 | Posted by in CARDIOLOGY | Comments Off on The Ventricular Volume Variability Study of the Pediatric Heart Network: Study Design and Impact of Beat Averaging and Variable Type on the Reproducibility of Echocardiographic Measurements in Children with Chronic Dilated Cardiomyopathy

Full access? Get Clinical Tree

Get Clinical Tree app for offline access