Summary
Background
Ranking of hospitals by lay media has attracted widespread attention but may not accurately reflect quality. Acute myocardial infarction (AMI) mortality is a straightforward measure of clinical outcome frequently used by ranking algorithms.
Aims
Our aim was to assess whether ranking among top hospitals correlated with lower in-hospital risk-adjusted mortality following admission for AMI.
Methods
Using a hierarchical regression model and the comprehensive nationwide database of hospital AMI admissions from 2004 to 2007 in France, we analysed crude and risk-adjusted hospital mortality rates in the ranked (‘best’) hospitals versus non-ranked hospitals. We subsequently restricted the comparison to non-ranked hospitals with matching on-site facilities.
Results
We analysed 192,372 admissions in 439 hospitals, 43 of which were in the ranked group. Patients admitted to the 396 non-ranked hospitals tended to be older with more comorbidities and underwent fewer revascularization procedures than patients admitted to ranked hospitals. Between hospital differences accounted for 10% of differences in mortality. Crude mortality was lower in ranked versus non-ranked hospitals (7.5% vs. 11.9%; P < 0.001). The survival advantage associated with admission to ranked hospitals was reduced after adjustment for age and sex (5.7% vs. 6.4%; P = 0.087) and comorbidities (4.9% vs. 5.5%; P = 0.102).
Conclusions
Ranked hospitals have similar adjusted AMI mortality rates to those not ranked and patient characteristics rather than hospital differences account for the variation in outcomes.
Résumé
Contexte
Le classement des hôpitaux dans la grande presse bénéficie d’un grand retentissement mais ne reflète pas nécessairement la qualité des soins. La mortalité hospitalière à la phase aiguë de l’infarctus du myocarde est un indicateur de résultat souvent utilisé.
Objectifs
Notre objectif était de vérifier l’association entre un bon classement des services par la presse et la mortalité ajustée de l’infarctus du myocarde.
Méthodes
À partir de la base nationale PMSI pour les années 2004 à 2007, nous avons construit un modèle hiérarchique de prédiction de la mortalité à la phase aiguë de l’infarctus du myocarde et comparé dans un premier temps les hôpitaux classés aux hôpitaux non classés. Dans un second temps nous avons restreint la comparaison aux hôpitaux classés et non classés ayant un niveau d’équipement comparable.
Résultats
Nous avons analysé 192 372 séjours dans 439 hôpitaux, parmi lesquels 43 figuraient dans le classement des 50 meilleurs. Les patients traités dans les 396 hôpitaux non classés étaient plus âgés, avec plus de comorbidités et des taux de revascularisation inférieurs. Le facteur « hôpital » expliquait 10 % de la mortalité. La mortalité brute était plus basse dans les hôpitaux classés (7,5 % vs 11,9 % ; p < 0,001). L’avantage de survie pour les patients admis dans les hôpitaux classés diminuait après ajustement sur l’âge et le sexe (5,7 % vs 6,4 % ; p = 0,087) et les comorbidités (4,9 % vs 5,5 % ; p = 0,102).
Conclusions
La mortalité dans l’infarctus du myocarde est liée aux caractéristiques du patient, le classement des hôpitaux parmi les 50 meilleurs ne prédit pas une meilleure survie.
Background
Public reporting of hospital performance is a current policy used by several governmental entities to compare hospital-specific or physician-specific outcomes. California , Pennsylvania , Scotland and Ontario, Canada , for example, have released hospital-specific reports for mortality following admission for acute myocardial infarction (AMI). While the regulatory and financing institutions promote the use of access measures and disseminate accreditation reports, the lay press has developed an annual hospital ‘star rating’ system that has gained wide public acceptance.
In France, as well as in other countries, the criteria used in lay press ranking differ from those used for institutional public reporting. The yearly ranking by ‘Le Point’ magazine is the most widely circulated comparison of hospitals in France. Le Point ranks the 50 ‘best’ hospitals by speciality; the rank of hospitals that are not among the top 50 is not disclosed further. The sales volume of the annual edition of Le Point on hospital ranking has always been the highest, except for presidential election editions. This success probably stems from the simplicity of the one-dimensional measure in the ranking system, contrary to the multidimensional and narrative assessments published by academic and governmental institutions. Additionally, the Le Point ranking is performed by condition, on a limited number of conditions or procedures (e.g. AMI, hip or knee replacement, cataract surgery and prostate cancer). Results are presented as a list of ‘the best hospitals for AMI care’ followed by another list of ‘the best hospitals for hip replacement’.
Our objective was to assess the robustness of this ranking. We selected AMI for the following reasons: while patients who have an AMI in France are referred via the emergency ambulance service and do not usually have the choice of a hospital and therefore would not use the magazine ranking, AMI care is a proxy for cardiac care in general. In the USA, top-ranked hospitals for AMI were found to provide evidence-based care for both acute and non-acute coronary syndromes and more generally to deliver care of better quality. We examined whether being ranked in the top 50 hospitals for care of AMI on the Le Point list was associated with lower risk-adjusted in-hospital mortality .
Methods
Using the comprehensive French nationwide hospital database, we examined crude and adjusted hospital mortality following admission for AMI among the 50 top-ranked hospitals on the Le Point hospital rating list and compared it with non-ranked French hospitals treating at least 10 AMI patients per year. We also performed a reverse analysis using the Le Point criteria to assess how well they predicted hospital mortality in the total AMI population.
Data source
The French medical information systems programme (PMSI) files for years 2004–2007 were used as the data source for this retrospective analysis. The PMSI file is an administrative database maintained by the Ministry of Health, containing all reimbursement claims submitted by acute-care hospitals (public, private not-for-profit and private for-profit, both teaching and non-teaching). The database records the following information: patient age, sex, date of admission, date of discharge, primary diagnosis, up to eight secondary diagnoses, up to six procedures, discharge status and total charges. Linkage of multiple admissions is possible. Out-of-hospital mortality, drugs on discharge and quality of life measures are not available. We also used information on the hospitals’ structures and equipment via the Annual Statistics of Health Institutions database maintained by the Ministry of Health (statistique annuelle des établissements de santé [SAE]).
Study population
Hospitals
The population in this study consisted of all French public, private not-for-profit and private for-profit hospitals with at least 10 discharges (each year) for an AMI, defined by a principal diagnosis code of I21–I22 of the International Classification of Diseases, Tenth Revision (ICD-10) during the years 2004–2007. Records were linked for transfers. The first step of the selection process was to exclude hospitals for which data on status were missing or incomplete; due to statistical analysis constraints (see below), the second step excluded hospitals with fewer than 10 yearly discharges for AMI. The comparison was repeated after restricting the analysis to hospitals with similar levels of technical equipment as those found in the top-ranked hospitals. Results of the selection process are presented in the flowchart in Appendix 1 , together with a description of hospitals and discharges excluded .
We extracted for each hospital: size (number of beds) and activity (number of yearly admissions for I21–I22 ICD-10 diagnoses); equipment available to treat cardiology patients (i.e. cardiac care unit, cardiac surgery and catheterization laboratory); teaching and ownership status.
Star rating system (Le Point rating)
The ranking methodology developed by Le Point magazine is available online . In short, the rank is a composite of case volume, notoriety, specialization, percentage of outpatient procedures, short length of stay, technical equipment and rate of hospital-acquired infections. The weight given to each item and the aggregation method are not disclosed. Only the top 50 ranked hospitals are listed and ranks for the non-listed hospitals are not disclosed: hospitals are therefore characterized by a dichotomous variable (ranked versus not ranked) instead of a continuous rank. As far as treatment of acute coronary syndromes was concerned, the list of ranked hospitals remained stable over our study period.
Patients
Admissions in the database were linked to obtain patient-level data. The primary purpose of the multilevel logit model was to predict the probability of in-hospital mortality during an admission for AMI, after controlling for patient and hospital characteristics. The following risk adjustments used a hierarchical modelling approach: age, sex, secondary diagnoses, previous AMI, tachycardia, congestive heart failure and renal failure ( Appendix 2 ).
Statistical analysis
A multilevel analysis was performed to estimate the expected probability of in-hospital mortality and identify the determinants of mortality. This approach was justified by the hierarchical structure of the data comprising two levels of analysis: the hospital level and the discharge/patient-level. Hierarchical models take into account both the variability between discharges and the variability between hospitals and avoid inappropriate conclusions related to residual variability when a source of variability is ignored . This method also allowed determination of the variability attributable to each level .
In order to obtain unbiased and accurate estimations using this type of model, a sufficient sample size at both the hospital and the patient/discharge levels is required, with one being dependent on the size of the other. In the absence of consensus regarding appropriate sample size, we followed recommendations from the literature in selecting hospitals with at least 10 yearly discharges on average to ensure a sample with at least 40 individuals per group.
A multilevel logit model was adopted because of the dichotomous nature of the dependent variable analysed (deceased versus alive). The model was used to identify a significant difference between hospitals with respect to mortality rates and to determine the patient and hospital characteristics associated with an increased probability of mortality. We subsequently compared the average in-hospital mortality rates of ranked hospitals versus non-ranked hospitals, first without adjustment and then after adjusting for age, sex and comorbid conditions. We then restricted the comparison to hospitals that were equipped with matching technical facilities.
We built a multilevel model to predict mortality based on the Le Point criteria applied to all hospitals and patients. Only the following variables were used (other variables such as ‘notoriety’ being unavailable to the public): cardiac care unit, cardiac surgery, catheterization lab, annual AMI discharges and percentage of angioplasty. We used backward selection because of correlation between the variables of interest.
Univariate differences were tested using the Wilcoxon signed-rank test. The comparison was repeated, focusing on the non-ranked hospitals presenting the same structural characteristics as the top-ranked hospitals (i.e. general and multidisciplinary hospitals equipped with at least one of three facilities: cardiac surgery, catheterization laboratory or cardiac care unit). Multilevel analysis was performed with HLM software (Scientific Software International, Lincolnwood, IL, USA). All other analyses were performed with SAS version 9.1 statistical software (SAS Institute, Cary, NC, USA).
Results
The study sample consisted of 192,372 patients treated in 439 hospitals over the 4-year period. Hospital and patient characteristics are presented in Tables 1 and 2 . Top-ranked hospitals were almost exclusively university hospitals with on-site revascularization facilities, admitting on average 400 AMIs each year. Compared with other hospitals, they admitted more AMIs ( Fig. 1 ) and their patients were younger, more frequently men, presented fewer comorbidities and were more likely to be treated with percutaneous coronary intervention (PCI). The average length of stay (2004–2007) was 6.9 ± 1.8 days for all patients, 7.0 ± 1.9 days for patients in non-ranked hospitals and 6.7 ± 1.2 days for patients in ranked hospitals.
| Variables | All ( n = 439) | Non-ranked ( n = 396) | Ranked ( n = 43) | P * |
|---|---|---|---|---|
| Type | ||||
| Public | 70.6 | 67.7 | 97.7 | |
| Private not-for-profit | 4.3 | 4.8 | 0.0 | < 0.001 |
| Private | 25.1 | 27.5 | 2.3 | |
| Status | ||||
| Public general hospital | 60.1 | 63.1 | 32.6 | < 0.001 |
| Private for-profit/not-for-profit general hospital | 26.2 | 29.0 | 0.0 | < 0.001 |
| Regional hospital | 9.8 | 3.8 | 65.1 | < 0.001 |
| Other | 3.9 | 4.0 | 2.3 | > 0.999 |
| Activity | ||||
| Occupancy rate (%) | 82.4 ± 10.6 | 82.0 ± 11.0 | 85.8 ± 5.3 | 0.012 |
| No. of full-time equivalent cardiologists | 4.1 ± 5.1 | 3.1 ± 3.7 | 13.6 ± 6.2 | < 0.001 |
| No. of yearly admissions for AMI | 109.6 ± 121.7 | 80.5 ± 77.9 | 377.4 ± 125.6 | < 0.001 |
| Equipment | ||||
| Cardiac care unit | 48.3 | 43.7 | 90.7 | < 0.001 |
| Cardiac surgery | 11.4 | 6.4 | 60.5 | < 0.001 |
| No. of catheterization laboratory units | 0.5 ± 0.8 | 0.4 ± 0.6 | 1.6 ± 0.9 | < 0.001 |
| Other | ||||
| LOS (2004–2007) (days) | 6.9 ± 1.8 | 7.0 ± 1.9 | 6.7 ± 1.3 | 0.348 |
| Study hospitals ranked in top 50: Le Point list 2006 | 9.8 | 0 | 100.0 | |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
