The aim of the present study was to externally validate the European System for Cardiac Operative Risk Evaluation (EuroSCORE) II (ESII) in patients undergoing percutaneous coronary intervention (PCI) and to compare its performance with that of its previously released versions, named additive (addES) and logistic EuroSCORE (logES). A total of 537 patients undergoing PCI were analyzed by different measurements of discrimination, calibration, and global accuracy. A significant gradient in all-cause mortality was seen with all the models at 30 days, 1 year, and 5 years, with the exception of the ESII at 30 days. The ESII had the lowest area under the receiver operating characteristic curve at all time points compared with its previous version, being 0.83 (vs 0.90 for both addES and logES) at 30 days, 0.75 (vs 0.82 for both addES and logES) at 1 year, and 0.69 (vs 0.77 for addES and 0.76 for logES) at 5 years. However, the ESII displayed a better calibration than the logES at 30 days, whereas both scores were miscalibrated at 1 and 5 years. The Brier score displayed similar global accuracy between the ESII and logES. In conclusion, the ESII is better calibrated than the logES at 30 days but does not represent a step forward in discrimination and global accuracy compared with its previous versions for predicting early- and long-term mortality of patients undergoing PCI.
We aimed at investigating whether the newly available European System for Cardiac Operative Risk Evaluation (EuroSCORE) II (ESII) may improve the prognostic ability of the previous logistic EuroSCORE (logES) and additive EuroSCORE (addES) models in patients undergoing percutaneous coronary intervention (PCI). Specific aims were to (1) externally validate the ESII for risk prediction in the context of PCI and compare its performance with those of its older versions and (2) evaluate the discrimination, calibration, and global accuracy of the ESII for all-cause mortality at 30-day, 1-year, and 5-year follow-up in patients with coronary artery disease undergoing PCI.
Methods
The study population included patients undergoing PCI with complete data for calculating the EuroSCORE (logistic and additive) and EuroSCORE II from the screening cohort of the Stenting of Renal Artery Stenosis in Coronary Artery Disease (RASCAD) study, a randomized clinical trial testing the effect of renal artery stenting versus medical therapy in patients with coronary and renal artery disease. The local ethics committee approved the use of clinical data for this study, and the requirement for informed written consent was waived on the condition that subjects’ identities were concealed. The investigators wrote the manuscript and are responsible for the completeness and accuracy of data gathering and analysis.
The 3 scores, logES, addES, and ESII were computed using a dedicated software by an investigator blinded to procedural data and clinical outcome. For the purposes of this analysis, the patient population was divided in 3 groups on the basis of each score tertiles. All data were assessed for quality and entered into a dedicated computerized database. Information about in-hospital outcome was obtained from an electronic centralized clinical database. After discharge, all clinical follow-up data were prospectively collected by scheduled clinic evaluations or direct telephone interviews. Referring cardiologists, general practitioners, and patients were contacted whenever necessary for further information. All repeat coronary intervention and rehospitalization data were prospectively collected during follow-up and entered into a centralized computer system or by directly contacting the hospitals in which the patients were admitted or referred. To maximize the quality ascertainment of the primary end point, administrative data on survival status were obtained.
For all analyses, a 2-sided p <0.05 was considered statistically significant. All data were processed using the Statistical Package for Social Sciences, version 18 (SPSS, Chicago, Illinois). Continuous variables are presented as mean ± SD or as median and interquartile range and were compared using Student’s unpaired t or Mann-Whitney rank sum tests, as appropriate. The normality assumption for continuous variables was evaluated by the Kolmogorov-Smirnov test. Categorical variables are presented as counts and percentages and were compared with the chi-square test when appropriate (expected frequency >5). Otherwise, Fisher’s exact test was used. Spearman’s test was used to assess the correlation between the scores. Cumulative rates of all-cause mortality were estimated by the Kaplan-Meier method, and the log-rank test was used to evaluate differences between groups. Although follow-up extended beyond 5 years in a proportion of patients at the time of data analysis, we restricted the follow-up to 5 years in all patients to account for bias introduced by incomplete follow-up. Patients lost to follow-up were considered at risk until the date of last contact, at which point they were censored.
To adjust for potential confounders, multivariate analysis of independent predictors of all-cause death was performed with a Cox proportional hazard regression model. The assumption of the proportional hazard was verified by a visual examination of the log (minus log) curves, and the linearity assumption was assessed by plotting the Martingale residuals against continuous covariates. To avoid over-fitting of the model and on the basis of the principle of parsimony validated by Ranucci et al, we entered a parsimonious set of potential predictors including age, left ventricular ejection fraction, and creatinine clearance as independent control variables and the 3 EuroSCORE models (logES, addES, and ESII) as the independent study variable of interest. Crude and adjusted hazard ratios and corresponding 95% confidence intervals were reported.
The performance of the EuroSCORE models was evaluated in terms of discrimination and calibration, as previously described. Briefly, discrimination is the probability that the score will assign higher values of risk to patients who will die compared with those who will not. It was measured using the receiver operating characteristic area under the curve, which ranges from 0.50 (no discrimination) to 1.0 (perfect discrimination). The comparison among curves was analyzed with the Delong method. In addition, discrimination was assessed using Somers’ D xy rank correlation between predicted probabilities and observed responses. When D xy = 0, the model is making random prediction, when D xy = 1, the predictions are perfectly discriminating. Calibration evaluates the degree of correspondence between the estimated probabilities produced by a model and the actual observation. For the logistic scores, it was measured by the Hosmer-Lemeshow test and by generating calibration plots that visually compare the prediction with the observed probability. The calibration plot is characterized by an intercept, which indicates the extent that predictions are systematically low or high, and a calibration slope that should be 0. The perfect calibrated predictions stay on the 45-degree line, whereas a segment below or above the diagonal, respectively, reflects overestimation and underestimation.
The global accuracy of the logistic models was finally tested calculating the Brier score (quadratic difference between predicted probability and observed outcome for each patient), an overall performance measure that consists of only positive values ranging from 0 (perfect prediction) to 1 (worst possible prediction), with lower scores indicating a greater accuracy.
Results
A total of 537 patients undergoing PCI were analyzed. The mean values of addES, logES, and ESII were, respectively, 4.7 + 2.6, 5.3 + 7.4, and 1.6 + 1.9. There was a weak correlation between the ESII and its previous version (r = 0.51 with addES and r = 0.49 with logES; p <0.001 for both). A total of 244 patients (45.4%) were differently categorized on the basis of the addES and the ESII, whereas a total of 248 patients (46.2%) were differently categorized on the basis of the logES and the ESII.
Baseline demographic, clinical and angiographic characteristics of the study population stratified by tertiles of addES, logES, and ESII are listed in Tables 1 and 2 . Patients in the highest tertiles of each score were older, more frequently women and more likely to present with hypertension, diabetes mellitus, peripheral artery disease, chronic obstructive pulmonary disease, previous coronary artery bypass grafting, and lower left ventricular ejection fraction compared with patients in the lowest tertiles. There were no significant differences in key angiographic and procedural characteristics across tertiles for each score, except that patients in the highest tertiles had a higher chance to remain with lesions untreated after PCI compared with patients in the lowest tertiles.
| Variable | addES | logES | ESII | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Low <3.0, n = 211 (%) | Middle 3.0–6.0, n = 208 (%) | High >6.0, n = 118 (%) | p | Low <2.3, n = 207 (%) | Middle 2.3–4.8, n = 153 (%) | High >4.8, n = 177 (%) | p | Low <0.9, n = 203 (%) | Middle 0.9–1.5, n = 173 (%) | High >1.5, n = 161 (%) | p | |
| Age, yrs ± SD | 56.4 ± 9.5 | 64.4 ± 8.6 | 71.4 ± 8.4 | <0.001 | 56.8 ± 9.4 | 63.5 ± 8.8 | 69.2 ± 9.2 | <0.001 | 55.6 ± 8.3 | 64.2 ± 9.5 | 70.4 ± 8.2 | <0.001 |
| Men | 184 (87.2) | 166 (79.8) | 73 (61.9) | <0.001 | 180 (87.0) | 124 (81.0) | 119 (67.2) | <0.001 | 182 (82.7) | 132 (76.3) | 109 (63.0) | <0.001 |
| Risk factors | ||||||||||||
| Hypertension | 165 (78.2) | 177 (85.1) | 109 (92.4) | 0.003 | 163 (78.7) | 126 (82.4) | 162 (91.5) | 0.002 | 158 (77.8) | 143 (82.7) | 150 (93.2) | <0.001 |
| Hypercholesterolemia | 165 (78.2) | 154 (74.0) | 97 (82.2) | 0.225 | 164 (79.2) | 113 (73.9) | 139 (78.5) | 0.444 | 159 (78.3) | 133 (76.9) | 124 (77.0) | 0.933 |
| Current smoker | 172 (81.5) | 144 (69.2) | 65 (55.1) | <0.001 | 166 (80.2) | 111 (72.5) | 104 (58.8) | <0.001 | 171 (84.2) | 110 (63.6) | 100 (62.1) | <0.001 |
| Diabetes mellitus | 58 (27.5) | 87 (41.8) | 58 (49.2) | <0.001 | 59 (28.5) | 60 (39.2) | 84 (47.4) | 0.001 | 30 (14.8) | 78 (45.1) | 95 (59.0) | <0.001 |
| Medical history | ||||||||||||
| Previous MI | 43 (20.4) | 60 (28.8) | 36 (27.1) | 0.061 | 44 (21.3) | 36 (23.5) | 59 (33.3) | 0.020 | 43 (21.2) | 38 (22.0) | 58 (36.0) | 0.002 |
| Peripheral artery disease | 2 (0.9) | 4 (1.9) | 17 (14.2) | <0.001 | 2 (0.9) | 2 (1.3) | 19 (10.7) | <0.001 | 2 (1.0) | 4 (2.3) | 17 (10.6) | <0.001 |
| COPD | 3 (1.4) | 10 (4.8) | 18 (15.2) | <0.001 | 1 (0.5) | 8 (5.2) | 22 (12.4) | <0.001 | 3 (1.5) | 8 (4.6) | 20 (12.4) | <0.001 |
| Previous PCI | 47 (22.3) | 62 (29.8) | 28 (23.7) | 0.184 | 49 (23.7) | 39 (25.5) | 49 (27.7) | 0.668 | 50 (24.6) | 44 (25.4) | 43 (26.7) | 0.903 |
| Previous CABG | 3 (1.4) | 10 (4.8) | 19 (16.1) | <0.001 | 4 (1.9) | 5 (3.3) | 23 (8.3) | <0.001 | 1 (0.5) | 3 (1.7) | 28 (17.4) | <0.001 |
| Previous stroke | 9 (4.3) | 16 (7.7) | 6 (5.1) | 0.302 | 8 (3.9) | 13 (8.5) | 10 (5.6) | 0.176 | 6 (3.0) | 13 (7.5) | 12 (7.5) | 0.092 |
| Clinical presentation | ||||||||||||
| Stable angina | 19 (9.0) | 10 (4.8) | 3 (3.5) | 0.040 | 19 (9.2) | 6 (3.9) | 7 (4.0) | 0.044 | 8 (3.9) | 13 (7.5) | 11 (6.8) | 0.295 |
| Unstable angina | 130 (61.6) | 146 (70.2) | 86 (72.9) | 0.062 | 127 (61.4) | 105 (68.6) | 130 (7.4) | 0.039 | 140 (69.0) | 119 (68.8) | 103 (64.0) | 0.539 |
| NSTEMI | 17 (8.1) | 21 (10.1) | 16 (13.6) | 0.282 | 14 (6.8) | 18 (11.8) | 22 (12.4) | 0.130 | 20 (9.9 | 10 (5.9 | 24 (14.9) | 0.021 |
| STEMI | 41 (19.4) | 29 (13.9) | 13 (11.0) | 0.096 | 42 (20.3) | 24 (15.7) | 17 (9.6) | 0.015 | 33 (16.3) | 30 (17.3) | 20 (12.4) | 0.427 |
| LVEF, % ± SD | 53.2 ± 6.8 | 49.2 ± 9.6 | 45.4 ± 11.9 | <0.001 | 53.3 ± 6.8 | 50.3 ± 9.1 | 45.7 ± 11.4 | <0.001 | 53.8 ± 6.7 | 50.7 ± 8.6 | 44.3 ± 11.2 | <0.001 |
| Variable | addES | logES | ESII | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Low <3.0, n = 211 (%) | Middle 3.0–6.0, n = 208 (%) | High >6.0, n = 118 (%) | p | Low <2.3, n = 207 (%) | Middle 2.3–4.8, n = 153 (%) | High >4.8, n = 177 (%) | p | Low <0.9, n = 203 (%) | Middle 0.9–1.5, n = 173 (%) | High >1.5, n = 161 (%) | p | |
| Number of lesions, n ± SD | 1.8 ± 1.0 | 1.9 ± 1.1 | 2.1 ± 1.2 | 0.070 | 1.8 ± 1.0 | 1.9 ± 1.2 | 2.0 ± 1.1 | 0.117 | 1.8 ± 1.0 | 1.9 ± 1.1 | 2.0 ± 1.1 | 0.300 |
| Bifurcations | 23 (10.9) | 22 (10.6) | 22 (18.6) | 0.071 | 23 (11.1) | 15 (9.8) | 29 (16.4) | 0.147 | 22 (10.8) | 27 (15.6) | 18 (11.2) | 0.317 |
| Treated vessels, n ± SD | 1.5 ± 0.8 | 1.5 ± 0.7 | 1.5 ± 0.6 | 0.924 | 1.5 ± 0.8 | 1.5 ± 0.7 | 1.5 ± 0.6 | 0.735 | 1.5 ± 0.8 | 1.5 ± 0.8 | 1.4 ± 0.5 | 0.322 |
| Treated lesions, n ± SD | 1.6 ± 0.9 | 1.6 ± 0.8 | 1.5 ± 0.7 | 0.876 | 1.6 ± 0.9 | 1.6 ± 0.9 | 1.6 ± 0.8 | 0.910 | 1.6 ± 0.8 | 1.6 ± 0.9 | 1.5 ± 0.7 | 0.235 |
| Untreated lesions, n ± SD | 0.2 ± 0.6 | 0.3 ± 0.7 | 0.6 ± 0.9 | 0.001 | 0.2 ± 0.5 | 0.4 ± 0.7 | 0.4 ± 0.9 | 0.016 | 0.3 ± 0.5 | 0.3 ± 0.7 | 0.5 ± 0.9 | 0.001 |
| Vessel treated by PCI | ||||||||||||
| Left main | 9 (4.3) | 12 (5.8) | 10 (8.5) | 0.292 | 9 (4.3) | 7 (4.6) | 15 (8.5) | 0.169 | 7 (3.5) | 10 (5.8) | 14 (8.7) | 0.103 |
| Left anterior descending | 122 (58) | 127 (61.1) | 64 (54.2) | 0.479 | 116 (56.0) | 97 (63.4) | 100 (56.5) | 0.315 | 112 (55.2) | 111 (64.2) | 90 (56.0) | 0.162 |
| Left circumflex | 64 (30.3) | 58 (27.9) | 42 (35.6) | 0.347 | 63 (30.4) | 43 (28.1) | 58 (32.8) | 0.656 | 64 (31.5) | 54 (31.2) | 46 (28.6) | 0.809 |
| Right coronary artery | 83 (39.3) | 73 (35.1) | 45 (38.1) | 0.658 | 85 (41.1) | 49 (32.0) | 67 (37.9) | 0.214 | 87 (42.9) | 57 (33.0) | 57 (35.4) | 0.115 |
| Implanted stent, n ± SD | 1.8 ± 1.2 | 1.8 ± 1.1 | 1.8 ± 1.0 | 0.884 | 1.8 ± 1.2 | 1.8 ± 1.2 | 1.8 ± 1.0 | 0.950 | 1.8 ± 1.2 | 1.9 ± 1.2 | 1.7 ± 1.0 | 0.198 |
| Drug-eluting stents, n ± SD | 1.7 ± 1.3 | 1.7 ± 1.2 | 1.6 ± 1.0 | 0.539 | 1.7 ± 1.3 | 1.7 ± 1.2 | 1.7 ± 1.0 | 0.889 | 1.7 ± 1.2 | 1.8 ± 1.3 | 1.6 ± 1.0 | 0.102 |
| Bare metal stents, n ± SD | 0.1 ± 0.4 | 0.1 ± 0.3 | 0.2 ± 0.5 | 0.085 | 0.1 ± 0.4 | 0.1 ± 0.3 | 0.2 ± 0.5 | 0.050 | 0.1 ± 0.4 | 0.1 ± 0.3 | 0.2 ± 0.5 | 0.114 |
Cumulative incidences of all-cause death corresponding to different tertiles of addES, logES, and ESII at 30 days, 1 year, and 5 years are listed in Table 3 . Corresponding Kaplan-Meier curves are depicted in Figure 1 . A significant gradient in the distribution of all-cause mortality was consistently seen with all the scores at each time point, with the exception of the ESII at 30 days. However, after adjustment, only the addES (hazard ratio: 1.18, 95% confidence interval: 1.03 to 1.35, p = 0.20) was shown to be independently associated with the 5-year risk of all-cause mortality, whereas the logES and the ESII were not. The same finding was noted when categorizing the 3 scores as tertiles. Age was an independent predictor of 5-year mortality regardless of the score entered in the multivariate model.
