Patients and Institutions

In this 36-month, prospective, multicenter cohort study (July 1, 1999, to December 31, 2001), all infants aged ≤ 6 weeks who were referred to one of the participating tertiary care institutions with the diagnosis of “heart murmur” or “rule out congenital heart disease” were enrolled by a pediatrician or neonatologist, who evaluated the patients and made all management decisions. Sonographers employed by community hospitals who scanned adult and pediatric patients performed all echocardiographic studies. Each participating tertiary care center interpreted echocardiograms obtained at delivery hospitals that had access to telemedicine and those that did not have access to telemedicine. Subjects born at hospitals with access to telemedicine constituted the study group, and those without telemedicine constituted the population from which the control group was drawn. Institutional review board approval was obtained at each participating institution (tertiary care and referral delivery hospitals).

A brief description of telemedicine co-investigators, facilities, technologies used, and number of transmissions per month per site is provided in Appendix A . In this study, no attempt was made to compare results by technology used. Each site co-investigator was responsible for study and control group patient identification and for data entry at that individual site.

Each participating tertiary care center provided both urgent and elective consultative services to both the telemedicine outreach sites and the control outreach sites. Participants fell into three general groups: (1) those needing immediate stabilization and referral for significant cardiac disease, (2) those who could be referred to an outpatient clinic at a later date for stable or asymptomatic cardiac disease (e.g., small ventricular septal defect, atrial septal defect, mild valvular stenosis), and (3) those with no heart disease. One control patient was matched to each study group patient within the referral population of each participating center by weight, gestational age, and diagnosis. For a large percentage of cases, it was not feasible to match on distance, and hence this aspect of matching was not successful in eliminating differences between groups on this variable. Every effort was made to ensure control group recruitment prospectively. However, in some cases, when the incidence of a specific diagnosis was very rare, a control patient was recruited retrospectively from infants referred in the 1-year time period before the start of the study. In addition to this stratified matching, we controlled for any residual influence of these variables by including weight and gestational age as covariates in all tests of between-subjects and within-subject effects.

Obtaining data from and matching control infants with mild or no heart disease, and matching them with infants in the telemedicine arm of the study, was more difficult compared with the infants with serious heart disease, who were always referred to the tertiary care institution for inpatient admission. We dealt with this problem by ensuring that co-investigators at each participating tertiary care center identified a collaborator at each of their referring institutions who worked with the co-investigator to facilitate data collection.

Participating physicians from nine institutions entered 1,057 patients into the database between September 1999 and December 2001. Of these, 581 (55.0%) received telemedicine services, and 476 (45.0%) were control patients. Control and experimental data were satisfactory from six of the nine institutions (see Appendix A ); the other three institutions did not enter control data, and hence their patients were not included in this analysis. For the remaining six sites, all telemedicine patients were matched with control subjects on diagnosis and disease severity. This yielded 37 matched telemedicine-control pairs with significant heart disease and 337 matched pairs with mild or no heart disease ( Table 1 ) from the six institutions with complete data. The length of time of telemedicine practice ranged from 3 to 8 years at these six hospitals. Because the nature of the questions we could address varied by severity of disease (e.g., the effects of telemedicine on avoidance of more intense and invasive treatment than necessary), we did not include the children with significant disease. Hence, the 337 matched pairs of infants with mild or no heart disease comprise the study sample for this report.

Data Collection

Data were collected at three separate time points for each study and control group patient. A single data entry Web site provided easy access for all participating centers to centralize electronic data entry. Data were entered for the study and control groups using questionnaires provided in the electronic database ( Appendices B–E ). Input from participating centers regarding refinement of these questionnaires was solicited at the beginning the study.

A master database received input from the participating centers, and aggregate data were provided to all principal and co-investigators every 3 months. In addition, each participating center received data specific to its own institution as requested. Aggregate data were presented at meetings of the investigators, which were held three times per year at national pediatric cardiology meetings (American Society of Echocardiography, American Heart Association, and American College of Cardiology). The data were masked to maintain confidentiality. A study code and key were maintained securely.

At the beginning of the study, each tertiary care center completed a detailed questionnaire ( Appendix B ) to ascertain the type of telemedicine technology at each site, the type of traditional cardiac care delivery, other demographic referral characteristics such as population served, and the volume of cardiac referrals at each participating tertiary care center. Data collected included patient identification number at both referring and tertiary care institutions, birth weight, gender, gestational age, and date of study entry.

Outcomes

The following outcomes were compared between telemedicine and control subjects: (1) fetal echocardiography diagnosis; (2) noncardiac diagnoses; (3) time from initial phone call requesting consultation to definitive diagnosis (interpretation of echocardiogram via telemedicine or videotape review from control sites); (4) transport to tertiary care center; (5) distance from tertiary care center; (6) total (combined for referral and tertiary care sties when applicable) LOS and intensive care unit (ICU) LOS (outliers were categorized as >60 days); and (7) adverse medical effects, including death, cardiac arrest, need for extracorporeal membrane oxygenation, intraventricular hemorrhage, mechanical ventilation, use of inotropic support, use of indomethacin, and use of prostaglandin E ₁ .

Statistical Analysis

The database was developed and maintained to ensure completeness and accuracy, with range limits and item verification for each variable. It was backed up securely on a regular basis. Access to the database Web site was password controlled, and patient-identifying information was not shared among institutions.

Diagnostic accuracy was approached through the use of sensitivity-specificity analysis and measures of concordance (Cohen’s κ statistic) in the telemedicine subgroup. The original echocardiographic videotape was the standard for accuracy. Only a subset of patients underwent follow-up echocardiography on the basis of clinical indications. Two-by-two tables were constructed (presence or absence of telemedicine diagnosis by presence or absence of interpretation of echocardiogram) for a number of diagnostic questions, calculating sensitivity, specificity, and positive predictive value for this population. Kappa statistics were used to assess the degree of agreement.

Descriptive statistics were computed as a further check on data integrity and to characterize the sample and outcomes. Because the telemedicine and control samples were matched, we expected no differences between groups with regard to patient risk factors, and analysis of variance (ANOVA) comparing groups confirmed this. Some research questions addressed differences between groups (telemedicine vs nontelemedicine), whereas others involved predictive models. For example, we were interested in between-subjects differences in time to definitive diagnosis, as well as in the factors influencing time to diagnosis for each group. We used general linear regression analysis, including ANOVA and analysis of covariance. The former type of question (i.e., the mean differences between groups) was tested using the F statistic (i.e., ANOVA or analysis of covariance), whereas the latter type of question (i.e., percentage of variance explained by individual variables) was evaluated using ordinary least squares regression. We corrected for the effects of inflated experiment-wise error using the Holm method. Holm’s approach addressed the potential problem posed by multiple comparisons, while applying less stringent criteria to the establishment of α levels than is the case for the Bonferroni method.

To estimate sample size, a priori statistical power calculations were carried out under several different assumptions regarding the distribution of variables to be used in the analysis. Some analyses were conducted on the entire sample using ordinary least squares regression. Testing the statistical power of multiple regression for sample sizes of 150 through 250, we determined that, controlling for four variables, if a given independent variable accounts for 3% of the variance in the dependent variable out of a total explained variance of 25%, with statistical significance (α) of 0.05, then statistical power (1 − β) is approximately 0.80 for a sample of 195 subjects. To test between-groups hypotheses, statistical power was estimated for ANOVA, assuming a small to medium effect size (0.3) and α = 0.05. The calculations suggested that power of 0.80 could be anticipated using a sample of 45 subjects; a medium effect size (0.5) could be detected with the same α level and power using a sample of 25. Hence, a priori power calculations supported the adequacy of the sample size in this study.