We sought to evaluate the ability of the Diamond and Forrester method (DFM) and the Duke Clinical Score (DCS) to predict obstructive coronary artery disease (CAD) on coronary computed tomographic angiography (CCTA) and the effect of these different risk scores on the appropriateness level using the 2010 Appropriate Use Criteria. Consecutive symptomatic patients who underwent CCTA for evaluation of CAD (n = 114) were classified as having a low, intermediate, or high pretest probability using the DFM and DCS. Using the Appropriate Use Criteria, the indications for CCTA were classified according to the pretest probability and previous testing. The CCTA results were classified as revealing obstructive (≥70% stenosis), nonobstructive (<70%), or no CAD. When the patients’ risk was classified using the DFM, 18% were low, 65% intermediate, and 17% high risk. When using the DCS, 53% of patients had a reclassification of their risk, most of whom changed from intermediate to either low or high risk (50% low, 19% intermediate, 35% high risk). The net reclassification improvement for the prediction of obstructive CAD was 51% (p = 0.01). Of the 37 patients who were reclassified as low risk, 36 (97%) lacked obstructive CAD. Appropriateness for CCTA was reclassified for 13% of patients when using the DCS instead of the DFM, and the number of appropriate examinations was significantly fewer (68% vs 55%, p <0.001). In conclusion, reclassification of risk using the DCS instead of the DFM resulted in improved prediction of obstructive CAD on CCTA, especially in low-risk patients. More patients were categorized as having a high pretest probability of CAD, resulting in reclassification of their examination indications as uncertain or inappropriate. These results identify the need for improved pretest risk scores for noninvasive tests such as CCTA and suggest that the method of risk assessment could have important implications for patient selection and quality assurance programs.
The Duke Clinical Score (DCS) and the Diamond and Forrester Method (DFM) represent the two most common methods for the evaluation of pretest probability of obstructive coronary artery disease (CAD) in symptomatic patients. These two scores were formulated using different statistical methods and also differ in the risk factors included, the cutpoints used to define obstructive CAD, and the definition of risk ranges ( Table 1 ). The appropriate use criteria (AUC) for cardiac computed tomography rely heavily on the pretest risk assessment to determine examination appropriateness. Although the original 2006 AUC for cardiac computed tomography suggested that symptomatic patients be evaluated using the DFM, the updated 2010 AUC acknowledged that this method does not include several important risk factors and concluded that “clinicians should become familiar with those [risk scores] that pertain to the populations they encounter most often.” Across major studies that have evaluated the clinical application of coronary computed tomographic angiography (CCTA), the score used to define subsets of risk in symptomatic patients has also been inconsistent. Recent data have suggested that the DFM might overestimate the pretest probability of CAD using both CCTA and invasive angiography. Given that the optimal method for pretest risk assessment before CCTA remains unknown, we sought to evaluate whether the DCS was superior to the DFM in predicting obstructive CAD in symptomatic patients referred for CCTA at a large medical center. We also sought to evaluate how the use of the DFM versus the DCS affected the level of appropriateness using the 2010 AUC.
| Variable | DFM | DCS |
|---|---|---|
| Formulation | Conditional probability analysis | Logistic regression analysis |
| Population | Combined symptomatic patients referred for invasive angiography and autopsy studies | Established and validated in symptomatic patients referred for invasive angiography |
| Prediction factors | Chest pain type; gender; age | Chest pain type; gender; age; previous myocardial infarction (with or without Q waves); smoking; hyperlipidemia; diabetes; ST-T wave changes |
| Risk score predicts | ≥50% stenosis | ≥75% stenosis |
| Risk categories | Low (<10%) | Low (<30%) |
| Intermediate (10–90%) | Intermediate (30–70%) | |
| High (>90%) | High (>70%) |
Methods
From March 2008 through July 2008, data were prospectively collected for 114 consecutive symptomatic patients who presented for CCTA for evaluation of CAD in their native coronary arteries at the Massachusetts General Hospital (Boston, Massachusetts). Scans performed for preoperative assessment and on research protocol subjects were excluded. Information, including patient demographics, baseline cardiac history, previous cardiac testing, and symptoms, was obtained from a self-administered questionnaire completed by each patient, an interview conducted by the physician performing the computed tomographic angiographic examination, the radiology order entry system, and the complete medical record. The institutional review board approved the study, and the need for informed consent was waived.
Baseline clinical data were used to identify patients as symptomatic and to determine their pretest probability of CAD. Patients were considered symptomatic if they had “chest pain syndrome,” defined according to the AUC as any symptoms thought consistent with obstructive CAD, including chest pain/tightness/burning, shoulder and jaw pain, or dyspnea. Typical angina was defined as chest discomfort that was (1) precipitated by physical exertion/emotion and (2) relieved with rest/nitroglycerin. Atypical angina was defined as chest discomfort that was associated with either of the 2 factors defining typical angina but not both, or dyspnea on exertion suspected as an anginal equivalent. Nonanginal chest pain was characterized as chest discomfort that lacked any of these associations. Patients were considered to have hypertension if they were taking antihypertensive medications or had been diagnosed with hypertension according to their medical records. The patients were considered to have hyperlipidemia if they were taking lipid-lowering agents or had been diagnosed with hyperlipidemia according to their medical records.
For each patient, the pretest probability of significant CAD was determined using the DFM and the DCS according to the original publications of these risk scores ( Table 1 ). Using the DFM, the patients were categorized as having low (<10%), intermediate (10% to 90%), or high (>90%) risk of obstructive CAD (defined as >50% luminal stenosis). The patients were then reclassified using the DCS as having low (<30%), intermediate (30% to 70%), or high (>70%) risk of obstructive CAD (defined as >70% luminal stenosis). Information regarding patients’ electrocardiograms (Q waves, ST-segment deviation) was not available for all patients and was not included in calculation of the DCS.
CCTA was performed on the Definition dual-source 64-slice CT scanner (Siemens Medical Systems). Clinical reporting was used to categorize the disease severity in each vessel. The overall disease severity was determined by the greatest stenosis identified among all evaluable segments:
Normal—absence of plaque and no luminal stenosis
Mild to moderate (nonobstructive) CAD—estimated stenosis <70%
Mild disease defined as stenosis estimated as <40%.
Moderate disease defined as stenosis estimated as ≥40% but <70%
Significant (obstructive) CAD—estimated stenosis ≥70%
The primary indication for each CCTA examination was determined from several sources:
Patient questionnaire: designed specifically for collection of data in the present study regarding chest pain and relevant medical history. The physician performing the scan reviewed the form with the patient to clarify any responses, if needed.
Radiology order entry system: the electronic radiology order entry system was used to determine the indications for the scan as selected by the ordering provider. These indications were often too broad to determine the primary examination indication as categorized using the AUC.
Electronic medical records: the medical record was used to collect additional patient data, such as previous testing, needed to establish the AUC indication. The most recent provider note was used to confirm the medical history provided by the patient and to identify the primary indication for the scan if that remained unclear.
As previously reported, two physicians (internist and cardiologist), who were unaware of the CCTA results, assigned each examination’s primary indication, and each study was categorized as appropriate, inappropriate, or uncertain using the 2010 AUC. When the specific indication for the examination was not addressed in the AUC or could not be determined, the designation of “not classified” was assigned. In rare cases, when the two physicians disagreed regarding the examination indication, all sources of information listed were reviewed jointly and the examination indication and appropriateness level decided by consensus.
A number of specific assumptions were required to apply the AUC in a standardized manner:
Examinations with multiple indications: a single patient could often be considered under multiple examination indications. To determine which was primary, the indications were considered as ordered in the hierarchy included in the 2010 AUC. In symptomatic patients referred for evaluation of native CAD, the hierarchy lists indications for acute coronary syndrome first, those that include previous testing next, and those determined by pretest probability alone last.
Previous cardiac testing: the period before CCTA in which previous cardiac testing results should determine examination appropriateness was not clearly specified in the 2010 AUC. CCTA was considered sequential to a stress imaging test (indications 22 and 23) if performed within 6 months. In contrast, other “previous” testing was considered if performed within 2 years (indications 20, 21, 24 to 26, and 29), or within 5 years among patients who were asymptomatic or with stable symptoms (indications 27 and 28).
Continuous variables are expressed as the mean ± SD, and differences in the mean values were assessed using Student’s unpaired t tests. Categorical variables are expressed as the frequency and percentage. For dichotomous variables, differences in the proportions were assessed using the chi-square test or Fisher’s exact test, as appropriate. McNemar’s test was used to assess for differences in proportions (i.e., between proportions of appropriate examinations with DFM vs DCS). Two-tailed P <.05 was considered statistically significant.
To compare the discriminatory power of the DCS and DFM, the area under the receiver operating characteristic curve was compared. For any two given patients, one with obstructive disease and one without, this statistic estimates how often each risk score will correctly differentiate between them. The proportion of patients reclassified using the DCS instead of the DFM was determined by constructing a reclassification table showing how many patients changed to a different category of pretest probability. However, because the proportion of patients who were reclassified does not assess whether the change in risk category was appropriate, we also estimated the net reclassification improvement using the method previously described by Pencina et al. When applied to our analysis, the net reclassification improvement estimates the extent to which those with obstructive CAD are appropriately reclassified up and those without obstructive CAD are appropriately reclassified down while subtracting the proportion of patients who have been inappropriately reclassified in each group. To examine the contribution of the subgroups with and without obstructive CAD to the net reclassification improvement, reclassification tables were constructed for patients with and without obstructive CAD ( Table 2 ). Data analysis was performed using Stata IC, version 10.0 (StataCorp, College Station, Texas).
| DFM | Total | |||
|---|---|---|---|---|
| Low | Intermediate | High | ||
| All patients ⁎ (n = 114) | ||||
| Duke Clinical Score | ||||
| Low | 20 | 37 | 0 | 57 (50%) |
| Intermediate | 1 | 18 | 3 | 22 (19%) |
| High | 0 | 19 | 16 | 35 (31%) |
| Total | 21 (18%) | 74 (65%) | 19 (17%) | |
| No or nonobstructive coronary artery disease † (n = 99) | ||||
| Duke Clinical Score | ||||
| Low | 20 | 36 | 0 | 56 (57%) |
| Intermediate | 1 | 16 | 2 | 19 (19%) |
| High | 0 | 13 | 11 | 24 (24%) |
| Total | 21 (21%) | 65 (66%) | 13 (13%) | |
| Obstructive coronary artery disease ‡ (n = 15) | ||||
| Duke Clinical Score | ||||
| Low | 0 | 1 | 0 | 1 (7%) |
| Intermediate | 0 | 2 | 1 | 3 (20%) |
| High | 0 | 6 | 5 | 11 (73%) |
| Total | 0 (0%) | 9 (60%) | 6 (40%) | |
⁎ Percentage reclassified: 60/114 = 53%; net reclassification improvement: 51%.
† Percentage reclassified: 52/99 = 52%; net reclassification improvement: 27%.
‡ Percentage reclassified: 9/15 = 60%; net reclassification improvement: 24%.
Results
The baseline characteristics of 114 consecutive symptomatic patients referred for CCTA are listed in Table 3 . Most patients had either nonanginal chest pain or atypical angina, and fewer had typical angina ( Table 3 ). More than one-half had hypertension or hyperlipidemia.
| Characteristic | Total (n = 114) | Obstructive CAD | p Value | |
|---|---|---|---|---|
| No (n = 99) | Yes (n = 15) | |||
| Age (years) | 56.3 ± 13 | 55.7 ± 13 | 60 ± 13 | 0.29 |
| Men | 59 (52%) | 48 (49%) | 11 (73%) | 0.07 |
| Diabetes mellitus | 17 (15%) | 15 (15%) | 2 (13%) | 0.85 |
| Hypertension | 65 (57%) | 53 (53%) | 12 (80%) | 0.05 |
| Hyperlipidemia | 71 (62%) | 59 (59%) | 12 (80%) | 0.13 |
| Current smoking | 14 (12%) | 10 (10%) | 4 (27%) | 0.07 |
| Previous myocardial infarction | 5 (4%) | 0 (0%) | 5 (33%) | 0.001 |
| Patient symptoms | ||||
| Nonanginal chest pain | 42 (37%) | 41 (41%) | 1 (7%) | 0.01 |
| Atypical angina | 46 (40%) | 40 (40%) | 6 (40%) | 0.35 |
| Typical angina | 26 (23%) | 18 (18%) | 8 (53%) | 0.002 |
Using the DFM, greater than half (65%) of the patients were classified as intermediate risk ( Figure 1 and Table 2 ). Using the DCS, the risk was reclassified in 53% of patients, and far fewer patients were classified as intermediate risk (19%, p <0.001; Table 2 ). When the DCS was applied to the 74 patients who were intermediate risk using the DFM, only 24% remained intermediate risk, just as many became high risk (26%), and most became low risk (50%). The changes in risk level among the patients categorized as intermediate risk using the DFM were the vast majority of the reclassification (93%), because few patients in the low- and high-risk groups using the DFM changed categories ( Table 2 ).
