Highlights
-
•
Pre-PVR AI-ECG was predictive of post-PVR survival in patients with rTOF.
-
•
AI-ECG complements imaging biomarkers for PVR risk stratification.
-
•
AI-ECG may help physicians to safely defer PVR based on the patient’s risk profile.
ABSTRACT
Background
Optimal timing of pulmonary valve replacement (PVR) in repaired tetralogy of Fallot (rTOF) remains challenging. We hypothesized that pre-PVR artificial intelligence-enabled electrocardiogram (AI-ECG) may inform optimal PVR timing in rTOF.
Methods
rTOF PVR patients at Boston Children’s Hospital (BCH) and Toronto General Hospital (TGH) with analyzable ECGs ≤3 months pre-PVR were included. Patients undergoing PVR were propensity score-matched 1:1 to non-PVR patients. Patients were partitioned into risk tertiles based on pre-PVR AI-ECG probabilities of 5-year mortality: low-, intermediate-, and high-risk.
Results
The PVR cohort included 605 patients (504 at Boston Children’s Hospital (BCH), 101 at Toronto General Hospital (TGH); median age 20.3 [IQR, 13.6-32.0] years; median follow-up 7.5 [IQR, 4.7-10.6] years; 3.6% mortality). Pre-PVR AI-ECG risk probability was predictive of post-PVR mortality (c-index 0.77), outperforming an established imaging-based model benchmark (c-index 0.70). AI-ECG remained an independent predictor when added to the benchmark model ( P <.001) with a higher c-index of 0.84. Survival was similar between low- and intermediate-risk groups (97-98% 15-year survival; P =.6), with increased mortality for the high-risk group (83% 15-year survival; P =.009). The matched cohort demonstrated that PVR was associated with increased survival overall (HR 0.28 [95% CI, 0.13-0.60], P =.001). Exploratory analyses stratified by risk group tertiles showed survival benefit associated with PVR in the intermediate-risk (HR 0.10 [95% CI, 0.01-0.86]; P =.04) and high-risk (HR 0.3 [0.1-0.7]; P =.005) groups, but not in the low-risk group ( P =.8).
Conclusions
AI-ECG predicts post-PVR survival in rTOF patients with a PVR survival benefit in intermediate- and high-risk, but not low-risk, groups. AI-ECG may complement imaging biomarkers to determine rTOF PVR timing.
Background
Although early survival of patients with repaired tetralogy of Fallot (rTOF) in developed countries is excellent, increased rates of morbidity and premature death in adulthood persist as a result of sequelae of right ventricular (RV) outflow tract dysfunction. ,, To relieve the chronic RV volume or pressure overload in this growing rTOF population, pulmonary valve replacement (PVR) is increasingly performed.
The effect of surgical or transcatheter PVR on RV remodeling is well established; however, PVR criteria, timing, and risk stratification remain elusive, relying on resource-intensive imaging (eg, cardiovascular magnetic resonance [CMR]) and other biomarkers. For example, while CMR metrics are incorporated within all PVR criteria, ,,, QRS duration and exercise stress test metrics are variably included. ,,,, In addition, contemporary literature on preoperative PVR risk and timing is derived from surrogate composite outcomes (ie, death and sustained ventricular tachycardia) that have similarly relied on CMR metrics. Reliance on resource-consuming imaging biomarkers such as CMR is incompatible with serial risk assessments at each clinic visit, and improper PVR timing has adverse consequences. Specifically, delayed PVR until onset of irreversible electromechanical cardiomyopathy can lead to poor clinical outcomes, while premature PVR may lead to risks of morbidity and unnecessary reinterventions. Therefore, it is of great interest to identify a robust, readily available, and economical biomarker accessible at each clinic visit to guide optimal PVR timing in patients with rTOF.
Our recent work demonstrates the promise of artificial intelligence-enabled electrocardiogram (AI-ECG) as an inexpensive and ubiquitous method to risk stratify patients with rTOF, achieving a similar performance to traditional CMR biomarkers. In this study, we aimed to investigate whether this same AI-ECG model predictive of 5-year mortality could be leveraged as a preoperative biomarker in patients with rTOF undergoing PVR intervention. To do so, we analyzed rTOF patient data from 2 centers enrolled in the International Multicenter TOF Registry (INDICATOR) cohort to determine whether AI-ECG could predict post-PVR survival and inform optimal timing of PVR based on pre-PVR AI-ECG risk tertiles.
Methods
AI-ECG model development and training cohort
This study utilizes a convolutional neural network model that was previously trained on ECGs obtained at Boston Children’s Hospital (BCH) and tested on BCH and Toronto General Hospital (TGH) patients enrolled in the INDICATOR cohort to predict 5-year mortality. Specifically, this model was trained on all ECGs obtained in the cardiology clinic (with and without rTOF) at Boston Children’s Hospital between 1990 (the earliest available electronically stored ECG) through June 2018 (training set of 216,503 ECGs from 78,578 patients). INDICATOR patients were excluded from this training cohort. As previously shown, this model was validated internally and externally to predict 5-year mortality in INDICATOR rTOF patients. Details of patient selection, data retrieval, signal processing, model architecture, and training have been published. None of the patients used to train the model were included in this study.
Retrieved ECGs passing quality control were trimmed to 2048 samples (∼8 seconds) to facilitate conveniently working with convolutional neural networks. The 12 leads x 2048 ECG data samples were then input into the previously described architecture. The output of the last block was fed into a fully connected layer with a sigmoid activation function to predict 5-year all-cause mortality after an ECG. Hyperparameter tuning was performed on the training set to minimize average cross-entropy loss using the Adam optimizer, with final hyperparameters of kernel size 9, batch size 64, and learning rate 0.001.
AI-ECG as a preoperative predictor of survival after PVR
To assess whether AI-ECG is a preoperative predictor of post-PVR survival, the PVR cohort included rTOF patients from BCH and TGH enrolled in INDICATOR with electronically archived ECGs ≤3 months pre-PVR. This dataset was derived from our previous work, and included INDICATOR cohort ECGs from 2001 to 2016. Exclusion criteria were: (1) patients with PVRs outside the aforementioned timeframe; (2) patients who did not have an electronically archived ECG ≤3 months pre-PVR at BCH or TGH; and (3) patients with ECGs that did not pass quality control, as previously described. The ECG closest to PVR was used if a patient had multiple qualifying ECGs.
The primary predictor variable was pre-PVR AI-ECG probability of 5-year mortality, and the primary outcome was time-to-death after PVR. The predictor was transformed into a categorical variable by partitioning patients into risk tertiles (low-, intermediate-, and high-risk) based on pre-PVR AI-ECG probabilities of 5-year mortality.
Cox proportional hazards regression was used to evaluate the association between pre-PVR AI-ECG probability (a continuous variable) and time-to-death after ECG. AI-ECG probabilities were benchmarked to an established imaging-based model that incorporates age, RV ejection fraction (EF), and RV mass-to-volume ratio. Patients who did not experience death were censored at the time of last known follow-up. Kaplan–Meier methodology was used to estimate primary outcome event rates.
AI-ECG to inform timing of PVR
To assess whether AI-ECG may inform timing of PVR, the PVR cohort was compared with non-PVR rTOF cohort using 1:1 propensity score-matching.
A propensity score was used as a patient matching factor to adjust for baseline differences between PVR and non-PVR patients. Analogous to our prior work, the propensity score was constructed using a logistic regression model with PVR as the outcome and the following predictor variables: site, age at repair, repair era, age at ECG, RV end-diastolic volume index, RV end-systolic volume index, RV mass/volume ratio, RV EF, left ventricular EF, and QRS duration. AI-ECG probabilities of 5-year mortality were further included in the model to further match patients and adjust for baseline differences. This logistic regression model achieved a c-index of 0.85, which is similar to our prior work. Patients with and without PVR were matched 1:1 according to propensity score. A difference of <25% of the cohort’s propensity score standard deviation was required to match. Duplicate use of patients was not allowed during the matching process.
In the 1:1 matched cohort, Cox proportional hazards regression was used to evaluate the association between PVR and time-to-death. To evaluate for differential associations between the low-, intermediate-, and high-risk groups and PVR with respect to the primary outcome, Cox models were created with a test of interaction between AI-ECG risk group and PVR status. Estimation of PVR effect was also performed within subgroups for hypothesis generation, irrespective of the interaction test significance. Given the small sample sizes at follow-up beyond 10 years, the survival analysis and the Cox regression model for the subgroups were truncated to 10 years of follow-up.
Trending AI-ECG probabilities over time
In an exploratory analysis to evaluate temporal changes in AI-ECG after PVR, we assessed AI-ECG 5-year mortality probabilities as a function of time from PVR. First, we assessed the change in AI-ECG probability from pre-PVR to 6-18 months post-PVR using a paired t-test. Second, we assessed AI-ECG 5-year mortality probabilities across the lifespan as a function of time from PVR in 6-month bins. Within these 6-month bins, the median and interquartile range of AI-ECG probabilities from all PVR patients were reported. ECGs ≤3 months post-PVR were excluded from this analysis to avoid the confounders of immediate postoperative changes.
Model explainability
Median waveform analysis and saliency mapping were performed to explain model behavior as previously described. Briefly, median waveform analysis provides visual representations of low-, intermediate-, and high-risk ECGs. Saliency mapping identifies regions of an ECG that contribute most to model predictions by highlighting components of the ECG where a change in input (ie, ECG voltage) led to a change in prediction. The median waveform was generated for each risk group using the 25 ECGs with 5-year mortality probabilities closest to the respective median probability. Saliency maps were generated using the average of the 25 high-risk group ECGs.
Data availability and software
Requests for BCH data and related materials will be reviewed internally to clarify if the request is subject to intellectual property or confidentiality constraints. Shareable data and materials will be released under a material transfer agreement for noncommercial research purposes. Use of BCH and TGH data was approved by their respective Institutional Review Boards. The programming code used to perform the analyses is available upon reasonable request.
Results
PVR cohort baseline characteristics
Within the INDICATOR cohort, we identified 647 patients at BCH and 178 patients at TGH undergoing PVR. After excluding 220 patients (180 patients with a PVR outside the study window, and 40 patients without an available ECG ≤3 months pre-PVR), 605 patients comprised the PVR cohort ( n = 504 at BCH, n = 101 at TGH) ( Figure 1 ). Baseline characteristics of the excluded PVR patients are shown in Supplementary Table I .
STROBE diagram for the 1:1 propensity score-matched cohort.
Abbreviations: ECG, electrocardiogram; PVR, pulmonary valve replacement.
Table I
Baseline characteristics of the overall PVR cohort ( n = 518) stratified by AI-ECG score tertile.
|
Low
( n = 202) |
Intermediate
( n = 201) |
High
( n = 202) |
P -value | |
|---|---|---|---|---|
| Demographics | ||||
| Sex (male) | 128 (63%) | 124 (62%) | 96 (48%) | .002 |
| Additional cardiovascular anomaly (yes) | 97 (48%) | 101 (50%) | 121 (60%) | .040 |
| Genetic syndrome (yes) | 32 (16%) | 29 (14%) | 38 (19%) | .5 |
| Age at repair | 0.4 (0.2, 1.5) | 0.7 (0.2, 3.9) | 2.3 (0.5, 6.6) | <.001 |
| Repair type | <.001 | |||
| Transannular patch | 143 (71%) | 125 (62%) | 100 (50%) | |
| RV-PA conduit | 27 (13%) | 40 (20%) | 53 (26%) | |
| Other | 32 (16%) | 36 (18%) | 49 (24%) | |
| Atrial arrhythmia (yes) | 3 (1.6%) | 14 (7.6%) | 37 (22%) | <.001 |
| ICD (yes) | 9 (4.5%) | 10 (5.0%) | 26 (13%) | .001 |
| NSVT (yes) | 30 (16%) | 31 (17%) | 39 (23%) | .2 |
| Age at PVR | 17.9 (13.1, 23.9) | 19.7 (13.6, 30.6) | 26.6 (15.1, 40.7) | <.001 |
| Follow-up time (years) | 7.2 (5.0, 9.6) | 7.6 (4.7, 10.6) | 7.7 (4.1, 11.5) | .6 |
| Mortality | 3 (1.5%) | 2 (1.0%) | 17 (8.4%) | <.001 |
| Age at Death (years) | 42.3 (32.2, 43.2) | 36.1 (24.7, 47.5) | 40.5 (19.3, 46.4) | >.9 |
| ECGs | ||||
| Age at ECG | 17.8 (13.1, 23.8) | 19.7 (13.6, 30.4) | 26.5 (14.9, 40.6) | <.001 |
| QRS duration | 144 (124, 160) | 150 (130, 164) | 154 (136, 172) | <.001 |
| AI-ECG probability | 0.004 (0.003, 0.006) | 0.016 (0.012, 0.022) | 0.057 (0.042, 0.104) | <.001 |
| CMR | ||||
| Age at CMR (years) | 17.6 (13.0, 23.2) | 19.3 (13.2, 28.8) | 24.7 (14.9, 39.5) | <.001 |
| RVEDVi (mL/m 2) | 166.8 (142.8, 192.7) | 162.9 (133.5, 194.2) | 172.1 (131.8, 214.5) | .3 |
| RVESVi (mL/m 2) | 82.8 (66.8, 100.2) | 85.4 (66.8, 104.2) | 93.3 (69.0, 126.4) | .001 |
| RVEF (%) | 49.6 (45.6, 54.5) | 47.4 (42.7, 52.7) | 43.9 (36.8, 48.8) | <.001 |
| RV mass/volume (g/mL) | 0.20 (0.17, 0.24) | 0.21 (0.18, 0.26) | 0.22 (0.17, 0.28) | .021 |
| LVEDVi (mL/m 2) | 88.7 (77.4, 99.9) | 84.1 (73.7, 92.6) | 83.4 (71.5, 101.8) | .009 |
| LVESVi (mL/m 2) | 36.9 (30.8, 43.2) | 35.7 (30.4, 41.8) | 37.4 (29.4, 49.2) | .2 |
| LVEF (%) | 59.0 (53.7, 62.9) | 57.2 (53.1, 61.0) | 55.5 (50.8, 61.5) | .002 |
| BVGFI (%) | 49.3 (45.0, 54.0) | 46.3 (42.7, 51.9) | 43.1 (37.6, 47.8) | <.001 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
