Highlights
-
•
Head-to-head external validation of BMC2 and PROGRESS-CTO scores in CTO-PCI.
-
•
BMC2 showed excellent discrimination for in-hospital mortality in CTO-PCI.
-
•
PROGRESS-CTO demonstrated moderate accuracy for procedure-related complications.
-
•
The 2 scores provide complementary patient- and operator-focused risk assessment.
ABSTRACT
Background
Chronic total occlusion percutaneous coronary intervention (CTO-PCI) is a high-risk procedure where reliable risk prediction is essential. We compared the performance of 2 risk models (Blue Cross Blue Shield of Michigan Cardiovascular Consortium [BMC2] and Prospective Global Registry for the Study of Chronic Total Occlusion Intervention [PROGRESS-CTO]) for the prediction of in-hospital outcomes after CTO-PCI.
Methods
We retrospectively analysed 1,157 CTO-PCI procedures performed at a single tertiary center between 2019 and 2023. Primary endpoint was in-hospital mortality. Discrimination was assessed with the area under the curve (AUC) method and calibration with calibration plots. Model accuracy was further quantified by the Brier score.
Results
Major adverse cardiac and cerebrovascular events (MACCE) occurred in 3.0% ( n = 35), and mortality in 1.0% ( n = 11). The BMC2 score outperformed PROGRESS-CTO for mortality prediction (AUC 0.96 vs 0.71; P <.001). BMC2 achieved excellent prediction for acute kidney injury (AUC 0.93), dialysis (0.91), and stroke (0.84), while PROGRESS-CTO provided moderate accuracy for periprocedural myocardial infarction, pericardiocentesis (both AUC 0.71), coronary perforation (AUC 0.72) and MACCE (AUC 0.62). Calibration plots indicated better calibration for the PROGRESS-CTO model, while both models showed low overall prediction error (Brier score).
Conclusions
In CTO-PCI, BMC2 offers superior performance for in-hospital mortality prediction. Their complementary features suggest BMC2 as a powerful, data-demanding, and patient-focused tool, while PROGRESS-CTO as an easy-to-calculate, procedure-oriented model.
Background
Percutaneous coronary intervention for chronic total occlusion (CTO-PCI) remains one of the most technically challenging procedures within interventional cardiology. Advances in techniques, ,, algorithms, , operator expertise, and dedicated devices ,, have led to improvements in procedural success, and the procedure is currently indicated for the improvement of symptoms and quality of life. However, the overarching principle behind CTO-PCI remains the avoidance of additional harm, particularly given the uncertain prognostic benefit of the procedure in terms of hard clinical endpoints. As a consequence, the mitigation and prevention of procedural complications are of paramount importance, and significant progress has been made in this field.
In this context, preprocedural risk assessment may play a pivotal role, as prevention is consistently more effective than treatment once complications occur. Risk scores serve 3 major purposes: first, to optimize patient selection; second, to support shared decision-making by providing individualized risk–benefit estimates; and finally, to guide procedural planning by anticipating complications and informing preventive strategies.
Several predictive models have been developed to support this process in both CTO ,,, and non-CTO ,, interventions. Among PCI models in all-comers, the Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) score was developed as a machine learning–based tool. Importantly, BMC2 has recently demonstrated superiority over established logistic regression–based scores , in patients undergoing complex higher-risk PCI.
On the other side, CTO-specific scores have been designed to address the unique characteristics of this patient population. Among these, the Prospective Global Registry for the Study of Chronic Total Occlusion Intervention (PROGRESS-CTO) complication scores provide an estimate of clinically significant adverse events (death, coronary perforation, pericardiocentesis, myocardial infarction [MI], and major adverse cardiovascular and cerebrovascular events [MACCE]) by integrating angiographic and technical variables. ,
In this study, we directly compared the BMC2 and PROGRESS-CTO complication scoring systems in the setting of CTO-PCI, with the aim of evaluating their relative performance, clinical applicability, and complementary roles in risk stratification.
Methods
Study population
All patients who underwent CTO-PCI at the University of Washington Medical Center between January 2019 and December 2023 were considered for inclusion. As previously described, data were automatically entered into the institutional CathPCI database, which was constructed using customized SQL code to extract information from the hospital’s electronic health record system (Epic, Epic Systems, Verona, WI), the NCDR CathPCI Registry submissions, and other institutional databases. The dataset included demographic and clinical characteristics, laboratory and echocardiographic data, angiographic and procedural details, as well as complications and in-hospital adverse events. After identifying all CTO-PCI cases, an experienced CTO operator independently reviewed each procedure to assess angiographic characteristics and procedural strategies. This study was approved by the University of Washington Institutional Review Board (STUDY00016614). The requirement for informed consent was waived due to the retrospective nature of the study.
Risk models
The PROGRESS-CTO complication scoring system was developed to estimate the risk of periprocedural adverse events in patients undergoing CTO-PCI. It provides risk prediction for in-hospital MACCE, mortality, pericardiocentesis, and acute MI. Using data from more than 10,000 CTO-PCI procedures in the PROGRESS-CTO registry, the score was built through multivariable logistic regression and internally validated with bootstrapping. The models incorporated clinical, angiographic, and procedural variables including age, sex, left ventricular ejection fraction, prior coronary artery bypass grafting, atrial fibrillation, calcification severity, lesion stump morphology, and crossing strategy (antegrade dissection and re-entry, or retrograde). Weighted points were assigned to each factor according to their odds ratios, allowing simplified bedside calculation. In addition, the PROGRESS-CTO perforation score was specifically designed to estimate the risk of clinical coronary artery perforation during CTO-PCI. Derived from over 9,600 procedures in the PROGRESS-CTO registry, it was developed also using multivariable logistic regression and internally validated with bootstrapping. The score incorporates 5 independent predictors: age ≥ 65 years, moderate/severe calcification, blunt proximal cap, use of antegrade dissection and re-entry or the retrograde approach. Weighted points were assigned according to the magnitude of the odds ratios, yielding a simplified 0–5 risk score.
The BMC2 scoring system leverages machine learning to predict multiple in-hospital adverse events following PCI, including mortality, AKI, new-onset dialysis, stroke, major bleeding, and the need for transfusion. This model utilizes 23 variables and leverages an XGBoost machine learning algorithm, trained on data from the Blue Cross Blue Shield of Michigan Cardiovascular Consortium registry. The BMC2 score was developed using a learning cohort of 107,793 patients undergoing PCI, with external validation performed in 56,583 procedures, and demonstrated strong predictive discrimination.
Although both scores were designed to predict periprocedural complications, they differ substantially in scope, methodology, and outcomes. The BMC2 model was developed in an all-comer PCI population, leveraging machine learning to capture complex nonlinear interactions, whereas the PROGRESS-CTO complication score was specifically created for CTO-PCI using multivariable logistic regression to create an easy-to-calculate integer-based score. Moreover, the only overlapping endpoint predicted by both tools is in-hospital mortality, while the remaining outcomes differ. In this study, our aim was to externally validate both scoring systems in the setting of CTO-PCI and to perform a direct head-to-head comparison of their predictive performance for in-hospital mortality.
Study definitions and study endpoints
CTO were defined as a 100% stenosis with a thrombolysis in MI flow grade of 0 and an estimated occlusion duration of ≥ 3 months BMC2 and PROGRESS-CTO complication scores were calculated for each patient to assess complication risk prediction. The main endpoint of this analysis was in-hospital mortality. All outcomes were defined according to the criteria reported in the original studies, , with the exception of MACCE, which was defined as a composite of all-cause mortality, MI, stroke, or pericardiocentesis. Urgent repeat revascularization was not included, in contrast to its incorporation in the PROGRESS-CTO complication score, as it was not captured by our institutional database. Additional outcomes, assessed only for the BMC2 model, included AKI (defined as an absolute increase in post-PCI creatinine of ≥ 0.5 mg/dl), new dialysis requirement (unplanned initiation of dialysis during hospitalization due to post-PCI renal deterioration), stroke (ischemic, haemorrhagic, or unspecified), major bleeding (a drop in haemoglobin >5 g/dl between pre and post-PCI measurements), and transfusion (administration of at least 1 unit of packed red blood cells or whole blood following PCI). MI was defined according to the Society for Cardiovascular Angiography and Interventions criteria. Coronary perforation, as defined in the PROGRESS-CTO perforation score, was considered any perforation requiring treatment, including pericardiocentesis, covered stent implantation, or other therapeutic interventions.
Statistical analysis
The discriminative performance of each risk model was quantified using the area under the receiver operating characteristic curve (AUC). Comparisons of AUC values between the BMC2 and PROGRESS-CTO scores were conducted with the DeLong test for mortality prediction. For each score additional performance metrics, including accuracy, sensitivity, specificity, and positive and negative predictive values, were calculated. For these secondary metrics optimal cut-off points were derived within the study cohort using the Youden Index, whereas the primary comparison between models was based on discrimination (AUC). Finally, a bootstrap resampling procedure (1,000 iterations) was used to estimate 95% CIs for the AUC of each model and for the paired difference in AUC between models.
Model calibration was evaluated using calibration plots and error indices, while overall goodness of fit was tested with the Hosmer–Lemeshow statistic to assess agreement between predicted and observed event rates. Model accuracy was further quantified by the Brier score, with values approaching zero indicating more accurate probability estimates and values closer to one reflecting poor predictive performance. For variables strictly required for the calculation of either score, cases with missing data were excluded without imputation, as done by the original studies. Conversely, for optional predictors in the BMC2 model (total cholesterol, HDL cholesterol, haemoglobin, and left ventricular ejection fraction), missing values were replaced with the cohort median, in accordance with the methodology applied in the original derivation study.
Statistical analyses were performed using R v 4.1.3 (R Foundation for Statistical Computing) in RStudio environment, version 2022.07.02 (RStudio, PBC).
Results
Study population
Between January 2019 and December 2023, 4,679 PCI procedures were performed at our institution. Among them 1,412 (30.2%) were CTO-PCI and were used for the present analysis. Overall, 1,157 of 1,412 (81.9%) procedures had sufficient data for evaluation. All excluded patients were omitted due to missing data necessary for the calculation of both the PROGRESS-CTO and BMC2 scores. Specifically, the variables most frequently missing in the excluded observations were: age = 94; left ventricle ejection fraction = 94; body mass index (height and/or weight) = 99; creatinine level = 3; procedural systolic blood pressure = 49; CSHA scale = 154; patient status during the procedure = 2; diabetes status = 99; NYHA = 142; heart failure type = 205; tobacco use = 124; PCI indication = 100; stress test result = 105. A comparison between included vs excluded patients is provided in the Supplementary Table 1.
Baseline population characteristics are summarized according to MACCE status ( Table 1 ). The median age was 68 years (61-74), and 189 patients (16.3%) were female. The majority of procedures were performed electively (90.7%). Plaque modification devices were used in 17.5% of procedures, most frequently rotational atherectomy (7.7%), whereas intravascular lithotripsy was employed in 3.5%.
Table 1
Characteristics of the study population according to MACCE incidence.
| Characteristic | Overall, N = 1,157 | No MACCE group, N = 1,122 (97.0%) | MACCE group, N = 35 (3.0%) | P -value |
|---|---|---|---|---|
| Age | 68 (61, 74) | 68 (60, 74) | 72 (67, 77) | .012 |
| Female sex | 189 (16.3%) | 179 (16.0%) | 10 (28.6%) | .047 |
| Left ventricle ejection fraction (%) | 50 (45, 53) | 50 (45, 53) | 50 (34, 54) | .392 |
| PCI status | <.001 | |||
| Elective | 1,042 (90.7%) | 1,016 (90.9%) | 26 (74.2%) | |
| Non-elective | 115 (9.3%) | 106 (9.1%) | 9 (25.8%) | |
| CTO-PCI indication | .083 | |||
| Stable angina | 993 (85.8%) | 967 (86.2%) | 26 (74.2%) | |
| Acute coronary syndrome | 144 (12.4%) | 138 (12.3%) | 6 (17.1%) | |
| Other PCI Indication | 20 (1.7%) | 17 (1.5%) | 3 (8.6%) | |
| Left main PCI | 115 (9.9%) | 112 (10.0%) | 3 (8.6%) | .941 |
| Multivessel PCI | 182 (15.7%) | 175 (15.6%) | 7 (20.0%) | .592 |
| Plaque modification | 203 (17.5%) | 196 (17.5%) | 7 (20.0%) | .713 |
|
89 (7.7%) | 83 (7.4%) | 6 (17.1%) | |
|
8 (0.7%) | 8 (0.7%) | 0 (0.0%) | |
|
78 (6.7%) | 77 (6.9%) | 1 (2.9%) | |
|
41 (3.5%) | 41 (3.7%) | 0 (0.0%) |
Abbreviations: CTO, chronic total occlusion; MACCE, major adverse cardiac and cerebrovascular events; PCI, percutaneous coronary intervention.
The overall in-hospital MACCE rate was 3.0% (35 of 1,157). Among these events, in-hospital death occurred in 11 patients (1.0%), stroke in 8 (0.7%), clinically relevant coronary perforation in 24 (2.1%), pericardiocentesis in 17 (1.5%), and MI in 2 (0.2%) ( Table 2 ). AKI developed in 39 patients (3.4%), with 4 (0.3%) requiring new dialysis. Blood transfusion was required in 22 patients (1.9%), whereas major bleeding was infrequent (0.6%). As anticipated, risk scores were significantly higher in patients with MACCE compared with those without (Supplementary Table 2).
Table 2
Incidence of peri-procedural complications following CTO-PCI
| Characteristic | Overall, N = 1,157 | No MACCE group, N = 1,122 (97.0%) | MACCE group, N = 35 (3.0%) | P -value |
|---|---|---|---|---|
| MACCE components | 35 (3.0%) | N/A | 35 (100%) | <.001 |
|
11 (1.0%) | N/A | 11 (31.4%) | <.001 |
|
8 (0.7%) | N/A | 8 (22.9%) | <.001 |
|
2 (0.2%) | N/A | 2 (5.7%) | <.001 |
|
17 (1.5%) | N/A | 17 (48.6%) | <.001 |
| Coronary perforation | 24 (2.1%) | 8 (0.7%) | 16 (46%) | <.001 |
| AKI | 39 (3.4%) | 30 (2.7%) | 9 (25.7%) | <.001 |
| New requirement for dialysis | 4 (0.3%) | 2 (0.2%) | 2 (5.9%) | .005 |
| Major bleeding | 3 (0.6%) | 3 (0.6%) | 0 (0%) | .912 |
| Transfusion | 22 (1.9%) | 8 (0.7%) | 14 (40.0%) | <.001 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree