Comparative Performance of Machine Learning and Traditional Risk Scores in Predicting Adverse Events After Transcatheter Aortic Valve Replacement in Patients With Atrial Fibrillation

Highlights

  • ML models and risk scores showed similarly modest prediction of stroke and bleeding.

  • ML models highlighted TAVR procedure-related predictors absent from risk scores.

  • Larger external datasets are needed to refine and validate ML risk tools.

Patients with atrial fibrillation (AF) following transcatheter aortic valve replacement (TAVR) remain at risk of ischemic stroke (IS) and bleeding. However, traditional risk scores provide modest predictions of IS and bleeding in these patients. We aimed to develop machine learning (ML) models that predict IS, major gastrointestinal bleeding (MGIB), all clinically relevant bleeding (CRB), and net adverse clinical events (NACE) using data from patients in the ENVISAGE-TAVI AF trial. Ten ML algorithms were trained per outcome using nested cross-validation; the best-performing model (highest F1 score) was validated on a 25% holdout set. Model performance was compared with logistic regression models using CHA₂DS₂-VA or HAS-BLED. Among 1,377 patients, 41 had an IS, 83 had MGIB, 375 had CRB, and 255 experienced NACE. The predictive abilities of a linear discriminant analysis algorithm for IS (F1 score = 0.08) and CHA₂DS₂-VA (F1 score = 0.09) were similarly low, but numerically better than HAS-BLED (F1 score = 0.05). Prediction of MGIB was similarly low for a logistic-lasso algorithm (F1 score = 0.11), CHA₂DS₂-VA (F1 score = 0.09), and HAS-BLED (F1 score = 0.12). For CRB, the predictive performance of a Naïve Bayes algorithm (F1 score = 0.39) was similar to CHA₂DS₂-VA (F1 score = 0.38) and HAS-BLED (F1 score = 0.41). The predictive ability of a logistic regression algorithm for NACE (F1 score = 0.33) was numerically better than CHA₂DS₂-VA (F1 score = 0.22) or HAS-BLED (F1 score = 0.27). In conclusion, ML offered similar predictive ability to established risk scores for thromboembolic and bleeding outcomes among TAVR patients with AF.

Patients with atrial fibrillation (AF) undergoing transcatheter aortic valve replacement (TAVR) are at significant risk of both thromboembolic and bleeding complications. Clinical risk scores are routinely used to risk-stratify patients with AF as recommended by both the European and American guidelines for the management of AF. , The CHA₂DS₂-VA score estimates stroke risk, whereas the HAS-BLED score assesses the bleeding risk in patients with AF. , Nonetheless, while simple and widely adopted, the CHA₂DS₂-VA and HAS-BLED scores were derived from general AF populations rather than post-TAVR patients, who tend to be older and have more comorbidities. Machine learning (ML) could provide an alternative way to improve risk prediction by incorporating complex linear and nonlinear interactions in high-dimensional datasets. In this study, we aimed to (1) develop ML models using data from the ENVISAGE-TAVI AF trial to predict adverse thromboembolic and bleeding events in patients with AF after successful TAVR and subsequently (2) compare the performance of these ML models to 2 commonly used risk scores (CHA₂DS₂-VA and HAS-BLED scores) in terms of discrimination, calibration, and reclassification.

Methods

Study design

We used patient-level data from the ENVISAGE-TAVI AF trial (NCT02943785), a multinational, multicenter, prospective, randomized, open-label trial with blinded endpoint adjudication, designed to compare edoxaban with vitamin K antagonist (VKA) therapy in patients with AF following successful TAVR. The trial enrolled patients who were 18 years of age or older, with either pre-existent paroxysmal/persistent AF or new-onset AF after successful TAVR for severe AS. Patients with comorbidities associated with a high bleeding risk (i.e., history of intracranial hemorrhage, esophageal varices, and unresolved periprocedural complications, etc.) were excluded. A full description of inclusion and exclusion criteria is given in Supplementary Table 1 . A total of 1,426 patients were enrolled across international centers and randomized between 12 hours and 5 days after the TAVR procedure. , All patients were followed for a median of approximately 18 months. Patients were allocated to receive either once-daily oral edoxaban (60 mg or 30 mg based on dose-reduction criteria per local labeling) or a standard VKA regimen targeting an international normalized ratio of 2.0 to 3.0 (or 1.6-2.6 for patients aged ≥70 years in Japan). Concomitant antiplatelet therapy was permitted at the discretion of the treating physician. Randomization was stratified by anticipated edoxaban dose reduction. The maximum treatment duration was up to 36 months.

Clinical outcomes

We examined 4 clinical outcomes of interest: (1) clinically relevant bleeding (CRB), (2) ischemic stroke (IS), (3) major gastrointestinal bleeding (MGIB), and (4) net adverse clinical events (NACE). CRB was classified according to the International Society on Thrombosis and Hemostasis definition and included both major (clinically overt bleeding associated with a reduced hemoglobin level, blood transfusion, symptomatic bleeding at a critical site, or death) and clinically relevant nonmajor bleeding (any sign or symptom of hemorrhage that does not fit the criteria for major bleeding but does meet at least 1 of the following criteria: (1) requires medical intervention by a healthcare professional, (2) leads to hospitalization or increased level of care, or (3) prompts a face-to-face evaluation). , NACE was defined as the composite of death from any cause, myocardial infarction, IS, systemic thromboembolic event, valve thrombosis, or major bleeding. IS was defined in accordance with the Valve Academic Research Consortium-2 consensus as an acute episode of focal cerebral, spinal, or retinal dysfunction caused by infarction of the central nervous system tissue.

Model development and statistical analysis

This analysis utilized data from the safety cohort, which included patients who received ≥1 dose of the study drug. The dataset included 180 variables, selected by experts for their potential clinical relevance. After basic data transformations and cleaning, including the removal of homogeneous features and highly correlated variables, 155 variables remained in the final dataset. Since this dataset was derived from a randomized controlled trial, baseline characteristics were well balanced across the edoxaban and VKA arms. Therefore, we did not separate model development by treatment group. Instead, treatment allocation was included as a candidate predictor. Ten different ML algorithms (logistic regression, lasso regression, ridge regression, elastic net, linear discriminant analysis, Naïve Bayes, decision tree, random forest, XGBoost, and multilayer perceptron) were evaluated using nested cross-validation on a training dataset comprising 75% of the patients from the full dataset. The nested cross-validation involved 2 loops to prevent overfitting and bias in model selection. In the inner loop, hyperparameter tuning was performed using Bayesian optimization. In the outer loop, the models were scored using the F1 score to determine the best model for each endpoint. The F1 score was selected as the appropriate performance metric for imbalanced classification tasks. Finally, trained models were evaluated on a holdout test set comprising 25% of the patients not included in the training set. We also compared the performance of the best ML model for each endpoint to a logistic regression model that used a traditional risk score (CHA₂DS₂-VA and HAS-BLED) as its only input variable. The CHA₂DS₂-VA and HAS-BLED scores are widely used clinical tools to guide anticoagulation management in patients with AF. The CHA₂DS₂-VA score estimates the risk of stroke or systemic embolism by assigning points for congestive heart failure, hypertension, age ≥75 years (2 points), diabetes mellitus, prior stroke or transient ischemic attack (2 points), vascular disease, and age 65 to 74 years. In contrast, the HAS-BLED score evaluates the risk of major bleeding, with points assigned for hypertension, abnormal renal or liver function, prior stroke, history of bleeding, labile INR, age >65 years, and concomitant drug or alcohol use. Both scores share several overlapping factors (i.e., hypertension, stroke history, and age), but they differ in their primary purpose. Unlike CHA₂DS₂-VA and HAS-BLED, which are based on a small set of predefined clinical risk factors, our ML models incorporated over 150 routinely collected clinical, laboratory, and procedural variables. This broader input allows ML to explore nonlinear and higher-order interactions that are not captured by traditional scores and to identify predictors specific to the TAVR population.

To identify the most impactful variables for the outcomes of interest, SHAP (Shapley Additive exPlanations) was employed. SHAP values utilize a game-theoretical approach to quantify the importance of a variable in a prediction by assessing its contribution across various combinations of variables. The features were ranked using the mean absolute SHAP value from the holdout test set. A detailed explanation of the methods used for the derivation and optimization of the ML models was previously described.

Results

Patient characteristics

Baseline characteristics of the overall safety cohort are listed in Table 1 . The study population included 1,377 patients, with a mean age of 82.1 ± 5.4 years. Approximately half (52.2%) were men, and the majority (83.2%) identified as White. The mean body mass index was 27.7 ± 5.5 kg/m², with a mean creatinine clearance of 58.2 ± 24.1 mL/min. Cardiovascular and metabolic comorbidities were highly prevalent, including hypertension (91.4%), hypercholesterolemia (70.0%), diabetes (36.7%), and advanced heart failure (New York Heart Association class III or IV) in nearly half (44.5%) of the cohort. Baseline thromboembolic and bleeding risks were elevated, reflected by a median CHA₂DS₂-VA score of 4 (mean 4.5 ± 1.3) and a mean HAS-BLED score of 1.6 ± 0.8.

Table 1

Baseline characteristics

Parameter Overall N = 1,377
Age at enrollment, years, mean ± SD 82.1 ± 5.4
Sex, male 719 (52.2%)
Race
White 1,146 (83.2%)
Ethnicity
Hispanic or Latino 352 (25.6%)
Smoking status at baseline
Current 52 (3.8%)
Weight, kg, mean ± SD 75.3 ± 17.6
BMI, kg/m 2, mean ± SD 27.7 ± 5.5
CrCl, mL/min, mean ± SD 58.2 ± 24.1
Hemoglobin, g/L, mean ± SD 115.7 ± 77.8
AF 874 (63.5%)
AF paroxysmal 569 (41.3%)
AF persistent
>7 days but <1 year 158 (11.5%)
≥1 year 110 (8.0%)
AF permanent 518 (37.6%)
AF flutter 18 (1.3%)
Hypertension 1,258 (91.4%)
Mitral valve disease 599 (43.5%)
Diabetes mellitus 506 (36.7%)
Hypercholesterolemia 964 (70.0%)
COPD 199 (14.5%)
Prior MB or predisposition to bleeding 119 (8.6%)
Heart failure
NYHA III or IV 613 (44.5%)
History of stroke or TIA 233 (16.9%)
Prior PCI 354 (25.7%)
Prior MI 191 (13.9%)
Prior CABG 124 (9.0%)
HAS-BLED score, mean ± SD 1.6 ± 0.8
CHA 2 DS 2 -VA score, mean ± SD 4.5 ± 1.3
STS risk score, mean ± SD 4.9 ± 3.8
AC therapy prior to randomization 1,075 (78.1%)
TAVR devices
Balloon-expandable valve 631 (45.8%)
Self-expandable valve 449 (32.6%)
Other 297 (21.6%)
History of labile INR 108 (7.8%)
Treatment group
Edoxaban arm 693 (50.3%)
VKA arm 684 (49.7%)
Edoxaban baseline dose
30 mg 320 (23.2%)
60 mg 370 (26.9%)

Data are presented as n (%) unless otherwise noted. Treatment arm is the actual treatment arm received which, in the safety analysis set, differs to the randomized treatment arm in 1 patient who was randomized to VKA but received edoxaban.

Clinically relevant bleeding

Out of 1,377 patients, CRB was recorded in 375 patients (27.2%). The Naïve Bayes model ( Supplementary Table 2 ) provided the best predictive performance, with an F1 score of 0.39, closely matching HAS-BLED (F1 score = 0.41) and CHA₂DS₂-VA (F1 score = 0.38). Among the ML predictors, the absence of history of previous valve replacement before index procedure (no valve replacement) had the highest SHAP value, indicating its strong influence ( Supplementary Figure 1 ).

Only gold members can continue reading. Log In or Register to continue

Stay updated, free articles. Join our Telegram channel

Aug 8, 2026 | Posted by in CARDIOLOGY | Comments Off on Comparative Performance of Machine Learning and Traditional Risk Scores in Predicting Adverse Events After Transcatheter Aortic Valve Replacement in Patients With Atrial Fibrillation

Full access? Get Clinical Tree

Get Clinical Tree app for offline access