On September 16, 1977, Andreas Gruentzig performed the first coronary angioplasty on a 38-year-old businessman with unstable angina and a discrete proximal left anterior descending (LAD) artery lesion. The procedure was a success, and on subsequent angiograms at 10 years and 23 years, the patient continued to have a patent artery.^1,2 Gruentzig reported his first series of favorable results and immediately called for a prospective randomized trial comparing it to bypass surgery.^3,4 The field of interventional cardiology was born, and percutaneous transluminal coronary angioplasty (PTCA) quickly spread, with multiple operators in many countries gaining experience.

Today, percutaneous coronary intervention (PCI) is the dominant form of coronary revascularization. The practice of interventional cardiology has changed dramatically to include numerous diagnostic, pharmacologic, and technologic advances. The challenge for the practicing interventionalist is deciding when to adopt new therapies or technologies, in which patients and clinical situations to apply them, and at what cost. Fortunately, in keeping with Gruentzig’s initial emphasis, the field now requires rigorous scientific studies to evaluate these new therapies and technologies. As a consequence, the interventional cardiologist is faced with a large body of medical literature of varying quality.

Therefore, a framework for reading and evaluating the medical literature is necessary to make the best decisions for patient care. This practice, termed evidence-based medicine, is defined by Sackett et al⁵ as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.” Much of the general framework used in this chapter and in most of the current practice of evidence-based medicine has been adapted from the in-depth guide on evaluating and teaching the medical literature provided by Sackett et al.⁶ The goal of this chapter is to provide some tools and specific examples that help the reader become facile at reading and evaluating the medical literature, specifically clinical trials with a focus on interventional cardiology.

Sackett and colleagues^5,6 have identified 3 steps as integral to the practice of evidence-based medicine. They are as follows: (1) asking a clinical question and finding an answer, (2) reading the answer article, and (3) applying the results to individual patient care. The first step, asking a clinical question and finding an answer, is not addressed directly in this chapter. With the increase in electronic resources, including PubMed, MEDLINE, and additional online references, most clinicians have access to the medical literature. The majority of the discussion focuses on reading the clinical trial article. Finally, there is a section on applying the results to the care of the individual patient. Many of the principles used in this chapter for evaluating the medical literature can be found in detail in the JAMA User’s Guide to the Medical Literature series.^7-10

You are in the cardiac catheterization lab with a cardiology fellow seeing a 62-year-old businessman with hypertension. The patient recently presented with increasing angina, for which he was admitted to the hospital and ruled out for a myocardial infarction. He was treated with 300 mg of clopidogrel on the night of admission and is now referred for cardiac catheterization. After discussing the diagnostic and potential interventional procedure with the patient, the fellow asks you if it would appropriate to treat the patient with bivalirudin and provisional glycoprotein IIb/IIIa inhibitor (GPI), because she recalls reading a trial on its benefits. You tell the fellow that is a good question, and she should bring the REPLACE-2 trial to the next catheterization lab meeting for discussion. During your discussion with the fellow, you learn that another patient, a 70-year-old gentleman with an ST-segment elevation myocardial infarction is being brought to the catheterization lab. You ask the fellow to also bring the EUROMAX trial¹¹ to the next cath lab meeting.

To determine if an article answers a clinical question, a clearly defined clinical question is required. The clinical question is made up of 4 parts: the population or clinical problem, the intervention, the control, and the outcome of interest.¹²

The population or clinical problem should be defined as clearly as possible. For instance, in the case presented earlier, this could be a patient with unstable angina and obstructive coronary artery disease. The intervention should be easy to define in many of the clinical situations in interventional cardiology. These can range from pharmacologic to device therapies. Again, in the case of the 62-year-old gentleman above, the intervention is percutaneous intervention with bivalirudin and provisional GPI use. The comparison group may either be a placebo or the current standard of care. This also may be a pharmacologic therapy, interventional device, or usual standard care practices. In this case, the comparison group is heparin and GPI use. Finally, the outcome of interest can be many things. Often, the reader can determine if the outcome measure used in the clinical trial is of interest and clinically important. Death, myocardial infarction (MI), need for urgent target vessel revascularization, or bleeding might all be considered important outcomes for this clinical situation.

In this example, the clinical question could be phrased as, “In patients with coronary artery disease, does percutaneous intervention with bivalirudin and provisional GPI use compared to heparin and GPI use decrease death, MI, urgent target vessel revascularization, or bleeding?” Using these simple criteria, many clinicians will be able to glance quickly through article titles to determine if the question is addressed. Once this first step in reading a clinical trial is accomplished, the reader can move to step 2, determining if the results are valid.

Determining if the results reported in a clinical trial are valid is the most important and time-consuming step in reading an article. The goal of determining the validity of a study design is to ensure the results reported are accurate. If the experiment or the clinical trial was set up in an unbiased manner, then the results are believable. However, if there is a major flaw in the experimental design, then the reader may interpret the results cautiously and potentially disregard the results. Unfortunately, for many clinical questions about therapeutics, there is no perfect clinical trial design. Therefore, many trials have some aspects that are well designed and conducted and others that are not. The practice of applying the rigorous criteria for validity and determining each trial’s strengths and weaknesses helps lend weight to findings and ultimately helps makes clinical decisions. This ability to weigh different study results helps the reader practice the “judicious use of the best available evidence.”

In addition, understanding clinical trial design helps put future clinical trial results into the context of prior findings with an understanding of previous limitations in the medical literature. The criteria for evaluating the validity of a study depend on the type of study question: therapy, prevention, diagnosis, etiology, or harm. In this section, we review the validity methodology for therapy questions, as these are the most common questions addressed in the medical literature. However, the criteria for the other types of studies are also easily available for review in the JAMA series on using the medical literature.^10,13,14

Step 1

Assess Randomization

The first step in determining the validity of a clinical trial evaluating a therapy is to determine if the study was randomized and follow-up was complete. Randomization is a crucial step in evaluating a therapy and should be considered one of the cornerstones of good trial design. Randomization assures the equal distribution of both measured and unmeasured characteristics that may have an effect on the measured outcome. Not all methods of randomization are the same. Patients can be randomized to a therapy by simple, block, and stratified randomization methods.

Simple randomization is random allocation of either the experimental therapy or the control therapy to the patients. This is usually accomplished with computer-generated random numbers. However, in trials with smaller numbers of patients, there can sometimes be imbalances in the number of patients in each arm of the trial depending on where in the random number sequence enrollment is completed. For this reason, block randomization is used. Block randomization creates blocks (eg, of 4 or 6) in which there are equal numbers of patients receiving the experimental or control therapy. In this fashion, even if enrollment stops in the middle of a block, the imbalance between the numbers of patients in each group is limited.

Stratified randomization is used to achieve balance between important characteristics that may affect the outcome without losing the benefits of randomization. For example, the number of diabetic patients receiving an interventional therapy such as a type of stent may play an important role on the rate of restenosis, the outcome of interest. In this case, a separate randomization sequence would be developed for diabetic and nondiabetic patients. In this manner, the trial would continue to be randomized and have equal numbers of diabetic patients in each group.

The value of randomization cannot be emphasized enough, particularly in interventional cardiology trials. Numerous unmeasured variables that may affect the pharmacologic and device therapies used in addition to the experimental therapy are best controlled with randomization. Without randomization, observational cohorts must attempt to control for these factors by first identifying all of the variables and then applying statistical methods. For this reason, studies evaluating therapies without randomization must be seen as describing associations and not reflecting true effect. Several examples exist in which the findings from observational data were not confirmed in randomized controlled trials, the most notable of which is the effect of estrogen replacement therapy on cardiovascular outcomes.

Assess Loss to Follow-Up

After randomization is determined, the reader must focus on accounting for the patients in the trial. In accordance with the CONSORT guidelines for reporting randomized controlled trials, many trials now provide a diagram showing the flow of all patients in a clinical trial. This helps the reader quickly determine the dropout rate and follow-up of the study. If 500 patients are randomized to a particular interventional device and only 400 are seen in follow-up, then a potential for an unmeasured effect of the therapy exists. While no loss to follow-up would be ideal in preserving the benefits of randomization, this is seldom the case. Sackett et al⁶ have proposed a 5-and-20 rule of thumb, whereby <5% loss to follow-up leads to little bias and >20% loss to follow-up poses a serious threat to validity of the results. In trials where a considerable number of patients (~20%) are lost to follow-up, significant caution must be used in interpreting the results. Even if the patients are not able to undergo all follow-up exams and procedures, the investigators must be able to report the outcome of these patients. If these data are not available, one way to assess whether the loss to follow-up can affect a trial’s results is to assume the worst-case scenario for the missing patients and to see how that would affect the results presented.

Step 2

Look for Blinding

The second step in evaluating the validity of clinical trial design in testing a therapy is in determining blinding. The goal of blinding is to ensure that once randomized, patients get care that is similar outside of the studied or control therapy. Blinding has numerous potential benefits depending on the trial design and individuals or groups that have been blinded. Blinding may reduce biased psychological responses to interventions and increase rates of complying with trial regimen among patients; reduce differential withdrawal, dose adjustment, or interventions by trial investigators; and most importantly, reduce information bias while analysis of the data is performed.¹⁵

Blinding should be applied to as many potential parties as possible. In the case of pharmacologic therapies, the medical staff providing care, the patients, and the outcomes assessors can and should all be blinded. This becomes more difficult in trials evaluating devices in interventional cardiology. It may not be feasible to blind the operator to the type of stent or the use of an atherectomy device. However, the patients, the other medical staff, the clinicians who see the patients in follow-up, the interventionalists who perform repeat angiograms, and the outcomes assessors can all be blinded to the therapy.

In the example cases, the investigators in the REPLACE-2 trial went to great lengths to retain blinding.¹⁶ Prior to randomization, the interventionalist specified a preference for abciximab or eptifibatide as the GPI of choice for the intervention. Each patient then received 3 infusions. In 1 treatment arm, patients received bivalirudin, heparin placebo, and GPI placebo, and in the other treatment arm, patients received bivalirudin placebo, active heparin, and active GPI. A blinded activated clotting time (ACT) was then performed, and active drug was given if the ACT was less than 225 seconds. Finally, if the interventionalist determined that provisional GPI was required, the active bivalirudin arm received active GPI, while the heparin and GPI arm received placebo GPI. In this fashion, blinding of the patients and medical staff was retained, which helped ensure patients in both arms of the trial received similar care during the procedure and afterward. On the other hand, the EUROMAX trial was an open-label trial—both the investigators and the patients were aware of which treatment was being administered.¹⁷ To minimize reporting bias that may be associated with open-label design of trials, the EUROMAX trial had an independent clinical events committee to adjudicate events blindly using standardized end point definitions.

Compare Baseline Characteristics

The next step is to determine if the patients were similar at the start of the trial. Most clinical trials present the baseline characteristics of the patients in the first table. If randomization was successful, the patients should be similar with regard to most characteristics. However, if the sample size for a randomized trial is very large, even modest differences may yield statistical significance. In such instances, the reader must be able to differentiate statistical significance from clinical relevance. As mentioned earlier, some important characteristics that affect the outcome may have been stratified, thus ensuring equal numbers of patients in each group. In smaller trials, there are often small differences in the groups. However, we would caution against making any significant conclusions from these differences and would continue to lend weight to the process of randomization. At best, differences in the groups and direction of potential effect on the results should be noted prior to reading the results.