Highlights
-
•
We trained and validated a new AMI mortality model from a national GWTG–CAD Registry.
-
•
LightGBM achieved the highest AUROC 0.874 with excellent calibration across groups.
-
•
The new model outperformed legacy ACTION and GLMM in total and 2,023 validation cohorts.
-
•
Comorbidities, transport mode, and SVI added value beyond legacy inputs.
ABSTRACT
Background
Cardiovascular disease remains the leading cause of mortality worldwide, with acute myocardial infarction (AMI) contributing to over 100,000 deaths annually in the United States. Accurate risk stratification for in-hospital mortality is essential for guiding clinical decisions, improving outcomes, and optimizing hospital resources. However, existing models often rely on limited predictor sets, outdated data, and linear methods that may not reflect current clinical practice.
Objective
To develop and validate a contemporary in-hospital mortality risk model for AMI patients, incorporating clinical, demographic, and social determinants of health, and to compare performance against the legacy ACTION Registry–GWTG model.
Methods
We utilized data from the American Heart Association (AHA) Get with The Guidelines–Coronary Artery Disease (GWTG–CAD) Registry. Patients with AMI admitted between October 1, 2019, and December 31, 2022 (201,191 patients from 605 hospitals) were used to develop the in-hospital mortality prediction model. A total number of 70,302 patients admitted in 2023 served as an independent validation cohort. We incorporated 27 predictors and benchmarked against the legacy ACTION Registry–GWTG model. Subgroup and sensitivity analyses assessed model performance across sex, race/ethnicity, ST-elevation myocardial infarction status, and time period.
Results
The Light Gradient Boosting Machine (LightGBM)–based GWTG–CAD model achieved the highest discrimination (area under the receiver operating characteristic curve [AUROC] 0.874, 95% confidence intervals, 0.867-0.880) and superior calibration across subgroups, outperforming the ACTION Registry–GWTG model (AUROC 0.859, 95% confidence intervals, 0.852-0.867). Comorbidities, transportation method, and community-level socioeconomic factors contributed meaningful predictive value beyond traditional predictors. The generalized linear mixed model (AUROC 0.865) provided interpretable odds ratios and calibrated probability estimates suitable for risk-adjusted benchmarking and quality improvement.
Conclusion
The GWTG–CAD model suite advances AMI mortality prediction through 2 complementary approaches: the LightGBM model offers superior discrimination for identifying high-risk patients across diverse subgroups, while the generalized linear mixed model provides a transparent tool for institutional benchmarking and quality improvement. Broader external validation is needed before clinical deployment.
Graphical Abstract
Background
Each year, acute myocardial infarction (AMI) affects approximately 1 million Americans and remains a leading cause of death in both the United States and worldwide. Accurate and well-calibrated models for predicting in-hospital mortality after AMI are critical for clinical decision-making, patient counseling, resource allocation, and hospital benchmarking. Notably, hospitals participating in the Get With The Guidelines–Coronary Artery Disease (GWTG–CAD) Registry, one of the largest AMI registries in the United States, rely on such mortality prediction models to support quality improvement and cross-hospital comparisons. ,
While existing risk models were developed for in-hospital AMI mortality prediction, including the 2013 Acute Coronary Treatment and Intervention Outcomes Network (ACTION) Registry–GWTG model, and GRACE Risk Score, these models were developed using older data and traditional statistical methods that may not reflect current clinical practice or capture nonlinear relationships between predictors and outcomes. Shifts in population health, including the growing prevalence of obesity and diabetes and changes in hospital systems further limit their relevance. Moreover, many models continue to use race as a proxy for risk, despite growing consensus that race is a social construct and that social determinants of health offer more precise and equitable predictors. ,
To address these limitations, we developed a generalizable mortality risk model using both traditional statistical and machine learning models on contemporary data from the GWTG–CAD registry. The goal of the study was to (1) test whether the inclusion of a broader set of clinical predictors and social determinants of health improves predictive performance; (2) determine whether the use of machine learning methods improves model performance compared with traditional linear models; and (3) develop novel models to be used for risk adjustment in quality improvement efforts.
Methods
Data source
The American Heart Association (AHA) GWTG–CAD Registry is a nationwide voluntary quality improvement program in the United States. Program information and data elements collected in the case report form are available at: https://www.heart.org/en/professional/quality-improvement/get-with-the-guidelines/get-with-the-guidelines-coronary-artery-disease . Participating hospitals upload clinical data of consecutive patients admitted with ST-elevation myocardial infarction (STEMI) or NSTEMI. Because the data are primarily used at the local site for quality improvement, each participating hospital obtained either human research approval to enroll cases without individual patient consent under the Common Rule, or a waiver of authorization and exemption from subsequent review by its institutional review board (IRB). Advarra, the IRB for the AHA, determined that this study is exempt from IRB oversight. The data collection and coordination for GWTG programs are managed by IQVIA (Parsippany, NJ).
Study population
We included patients admitted to hospitals participating in the AHA GWTG–CAD Registry between October 1, 2019, and December 31, 2022, as the primary cohort for model development and internal testing. To evaluate models’ robustness and validity, we identified a separate validation cohort comprising patients admitted between January 1, 2023, and December 31, 2023. Data preprocessing followed the previously established methodology.
Eligible patients presented to the hospital with signs, symptoms, or complaints consistent with AMI (eg, chest pain, tightness in the chest, or shortness of breath), and received a final diagnosis of AMI (STEMI or NSTEMI). AMI diagnosis was defined by either (1) Troponin I or Troponin T results > the upper limits of normal, or (2) ECG results consistent with AMI. Patients with missing data on sex, race/ethnicity, or discharge status, as well as those transferred out of participating hospitals, were excluded due to incomplete outcome data. The detailed inclusion and exclusion criteria are shown in Supplementary Figure 1.
Outcome and predictors
The outcome was in-hospital mortality, defined as death from any cause during the index hospitalization.
Predictors included variables recorded at the time of presentation, including demographics, medical history, comorbidities, electrocardiogram findings, initial vital signs, laboratory values, and socioeconomic status, with a total of 27 variables (Supplementary Table 1), of which 9 were included in the legacy ACTION–GWTG risk score. Sex, race, and ethnicity were self-reported. Patient age at the time of diagnosis was calculated from the date of birth and the arrival date and time. Vital signs were values measured on first medical contact (FMC), by emergency medical services (EMS) for patients transported to the hospital by EMS, and in the emergency room for patients who transported themselves. Cardiac arrest prior to arrival indicates that the patient experienced an out-of-hospital cardiac arrest during prehospital care provided by EMS. Heart failure or cardiogenic shock on FMC was determined upon arrival at the hospital. Body mass index was calculated using the first weight and height obtained upon hospital arrival. The initial serum creatinine and initial troponin value were defined as the first values acquired during the index hospitalization. STEMI on ECG was determined from provider documentation that the first or subsequent ECG findings were consistent with STEMI or a STEMI equivalent. Markers of socioeconomic status included insurance and the social vulnerability index (SVI), which was determined based on patients’ residence. The SVI was calculated at the zip-code-level on the basis of 5-year estimates from the American Community Survey (2015–2020) and linked to individual-level participant data in the GWTG cohorts. The SVI ranges from 0 to 1, with higher values indicating greater social vulnerability. To handle missing data for predictors, we applied median imputation for variables with low missingness (initial serum creatinine, creatinine clearance, troponin levels, and SVI), as distributional distortion was minimal. For variables with greater missingness or stronger multivariable dependence (body mass index, systolic blood pressure [SBP], heart rate, smoking history, cardiogenic shock at FMC, cardiac arrest prior to arrival, and heart failure at FMC), we used iterative chained-equations imputation (scikit-learn’s IterativeImputer) to preserve joint relationships.
Model development
We developed prediction models using 2 complementary methodologies. First, we applied a generalized linear mixed (GLMM) model, incorporating hospital site as a random effect to account for site-level variability in outcomes. Such a model offers the advantage of interpretability, allowing for the estimation of adjusted odds ratios(ORs) and the inclusion of hospital-level effects. This modeling was also used in existing national benchmarks, such as the ACTION Registry–GWTG and GRACE models. , Second, we implemented a range of machine learning models, including random forest, Extreme Gradient Boosting Machine (XGBoost), and Light Gradient Boosting Machine (LightGBM). These tree-based ensemble methods are well-suited for clinical prediction tasks due to their ability to model complex, nonlinear interactions among variables. LightGBM and XGBoost, in particular, have demonstrated superior performance in a range of medical prediction studies. ,,
To benchmark our models, we also compared their performance to the ACTION Registry–GWTG model. For a fair comparison, we retrained the ACTION Registry–GWTG model using our training cohort, applying its published feature set and model structure. We then evaluated its performance on the validation dataset. Calibration performance was further examined across specific subgroups.
Statistical analysis
Summary statistics of baseline patient demographics and comorbidities, along with characteristics of the treating hospitals, were described using medians with interquartile range (IQR) and frequencies as appropriate. For descriptive analysis, we used the 2-proportion test for categorical values and used the Kruskal–Wallis test for continuous variables. For GLMMs, we calculated ORs with 95% CIs for each included variable and displayed results using a forest plot. For machine learning models, we used SHapley Additive explanations (SHAP) to identify the most influential factors.
To evaluate model performance, the overall study cohort (2019–2022) was split into training (80%) and validation (20%) sets using bootstrapping. Discrimination was assessed by the area under the receiver operating characteristic curve (AUROC), with 95% confidence intervals (CIs) estimated via 100 bootstrap iterations. Calibration was evaluated both in the overall cohort and across predefined subgroups.
Several secondary analyses were conducted. We first evaluated the model performance across subgroups stratified by sex, race/ethnicity, STEMI vs non-STEMI (NSTEMI) presentation, and year of diagnosis. Second, to assess generalizability in the postpandemic era, we tested model performance using a separate cohort of patients admitted in 2023. Third, we examined the impact of including race as a predictor by developing models with and without race. These models were built using different sets of input variables—from basic demographics (e.g., age and sex) to more comprehensive inputs that included clinical history, laboratory results, and social factors such as insurance status and SVI. For each comparison, we evaluated differences in AUROC to determine whether the inclusion of race improved model performance.
All analyses were conducted using R (v4.2.0) and Python (v3.7.16) on the AHA’s Precision Medicine Platform ( https://pmp.heart.org ). All P values are two-sided. Two-sided P values (significance threshold P <.01) were used for descriptive comparisons only, while model performance metrics were summarized using 95% CIs. This study was designed and reported in accordance with the TRIPOD statement, with relevant elements from the TRIPOD-AI extension incorporated to ensure transparency and reproducibility.
Results
Study population
From an initial dataset of 571,331 patients in the GWTG–CAD Registry, 305,779 were identified with discharge dates between October 1, 2019, and December 31, 2022. After applying exclusion criteria, 30,258 patients were removed due to missing demographic or clinical data (including race, ethnicity, gender, mode of transport, discharge status, or troponin units) or age outside the 18 to 100 year range. Furthermore, 27,483 patients were excluded because they were transferred out of the hospital, and 6,620 were excluded for lacking a primary cardiac diagnosis of STEMI or NSTEMI. This resulted in a final study cohort of 201,191 patients from 605 hospitals (Supplementary Figure 1).
Among the 201,191 patients who were hospitalized with AMI ( Table 1 , Supplementary Figure 1), the median age was 65 years (IQR: 57-75 years), and 32.7% of patients were female. Non-Hispanic White were 70.8%, non-Hispanic Black 13.6%, and Hispanic 8.1%. In-hospital death occurred in 10,389 (5.2%) patients. Compared to patients who survived, patients who died were older (median age 72 vs 65 years), more often female (39.1% vs 32.4%), had diabetes mellitus (36.4% vs 31.4%), and atrial fibrillation (12.9% vs 7.4%). Patients who died were more likely to have had cardiogenic shock (31.1% vs 2.8%) and cardiac arrest prior to hospital arrival (22.8% vs 2.1%). The held-out validation cohort consisted of 70,302 patients hospitalized between January 1 and December 31, 2023, with similar distributions of demographic characteristics, comorbidities, and social factors compared to the overall study cohort (Supplementary Table 2).
Table 1
Baseline characteristics.
| Variables |
Patient alive
( n = 190, 802) |
Patient died
( n = 10, 389) |
Total
( n = 201,191) |
P value |
|---|---|---|---|---|
| Age (years), Median (IQR) | 65 (18) | 72 (18) | 65 (18) | <.001 |
| Female ( n , %) | 61,762 (32.37) | 4,057 (39.05) | 65,819 (32.71) | <.001 |
| Race/ethnicities ( n , %) | ||||
| Non-Hispanic White | 135,231 (70.88) | 7,246 (69.75) | 142,477 (70.82) | .014 |
| Non-Hispanic Black | 26,099 (13.68) | 1,269 (12.21) | 27,368 (13.60) | <.001 |
| Hispanic | 15,412 (8.08) | 910 (8.76) | 16,322 (8.11) | .014 |
| Other | 14,060 (7.37) | 964 (9.28) | 15,024 (7.47) | <.001 |
| BMI, Median (IQR) | 28.98 (7.28) | 27.90 (7.70) | 28.91 (7.32) | <.001 |
| Medical history ( n , %) | ||||
| Diabetes mellitus | 59,916 (31.40) | 3,780 (36.38) | 63,696 (31.66) | <.001 |
| Hypertension | 123,435 (64.69) | 6,811 (65.56) | 130,246 (64.74) | .073 |
| Dyslipidemia | 98,033 (51.38) | 4,912 (47.28) | 102,945 (51.17) | <.001 |
| Smoking history | 64,386 (33.74) | 2,702 (26.01) | 67,088 (33.35) | <.001 |
| Currently on dialysis | 4,435 (2.32) | 544 (5.24) | 4,979 (2.47) | <.001 |
| Prior myocardial infarction | 31,187 (16.35) | 1,541 (14.83) | 32,728 (16.27) | <.001 |
| Prior percutaneous coronary intervention | 37,076 (19.43) | 1,764 (16.98) | 38,840 (19.31) | <.001 |
| Prior CABG | 15,005 (7.86) | 926 (8.91) | 15,931 (7.92) | <.001 |
| Atrial fibrillation | 14,110 (7.40) | 1,343 (12.93) | 15,453 (7.68) | <.001 |
| Cerebrovascular diseases | 17,007 (8.91) | 1,369 (13.18) | 18,376 (9.13) | <.001 |
| Peripheral arterial disease | 10,012 (5.25) | 844 (8.12) | 10,856 (5.4) | <.001 |
| Lab values at admission, median (IQR) | ||||
| Initial serum creatinine | 1.00 (0.40) | 1.40 (0.90) | 1.00 (0.4) | <.001 |
| Creatinine clearance | 80.39 (48.69) | 54.17 (46.96) | 80.39 (49.00) | <.001 |
| Troponin | 0.19 (1.20) | 0.69 (5.48) | 0.2 (1.31) | <.001 |
| STEMI on ECG ( n , %) | 87,355 (45.78) | 6,783 (65.29) | 94,138 (46.79) | <.001 |
| Heart rate at admission, Median (IQR) | 83 (27) | 87.67 (30) | 83 (27) | <.001 |
| Systolic blood pressure at admission, mm Hg, Median (IQR) | 150 (41) | 130 (45) | 149 (42) | <.001 |
| Heart failure FMC, ( n , %) | 21,803 (11.43) | 2,776 (26.72) | 24,579 (12.22) | <.001 |
| Cardiogenic shock FMC ( n , %) | 5,360 (2.81) | 3,233 (31.12) | 8,593 (4.27) | <.001 |
| Cardiac arrest prior to arrival ( n , %) | 3,942 (2.07) | 2,367 (22.78) | 6,309 (3.14) | <.001 |
| Documented LVEF, % | ||||
| <30% | 15,770 (8.27%) | 3,138 (30.21%) | 18,908 (9.40%) | <.001 |
| 30%-39% | 24,086 (12.62%) | 1,733 (16.68%) | 25,819 (12.83%) | <.001 |
| 40%-49% | 37,901 (19.86%) | 1,426 (13.73%) | 39,327 (19.55%) | <.001 |
| 50%-59% | 62,611 (32.81%) | 1,238 (11.92%) | 63,849 (31.74%) | <.001 |
| 60%-69% | 40,085 (21.01%) | 811 (7.81%) | 40,896 (20.33%) | <.001 |
| ≥70% | 1,830 (0.96%) | 89 (0.86%) | 1,919 (0.95%) | .320 |
| Insurance ( n , %) | ||||
| Medicaid | 23,348 (12.24) | 1,286 (12.38) | 24,634 (12.24) | .6791 |
| Medicare | 54,762 (28.70) | 4,119 (39.65) | 58,881 (29.27) | <.001 |
| Private/VA/CHAMPUS | 91,622 (48.02) | 3,881 (37.36) | 95,503 (47.47) | <.001 |
| Self-pay/No Insurance | 12,433 (6.52) | 602 (5.79) | 13,035 (6.48) | .004 |
| Not documented/UTD | 8,637 (4.53) | 501 (4.82) | 9,138 (4.54) | .166 |
| Transportation ( n , %) | ||||
| EMS– Air | 1,348 (0.71) | 111 (1.07) | 1,459 (0.73) | <.001 |
| EMS– Ground Ambulance | 75,423 (39.53) | 6,350 (61.12) | 81,773 (40.64) | <.001 |
| Walk-in | 70,591 (37.00) | 1,777 (17.10) | 72,368 (35.97) | <.001 |
| Transferred from another facility | 43,440 (22.77) | 2,151 (20.70) | 45,591 (22.66) | <.001 |
| SVI (0-1), Median (IQR) | 0.4346 (0.4033) | 0.4346(0.4050) | 0.4346 (0. 0.4039) | <.001 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
