Type of question
What are the benefits of this intervention and its relative safety?
Treatment benefits and common and predicable harms
Experimental studies: randomised controlled trials or n-of one trials
What are the harms induced by this intervention?
Experimental studies: n-of 1 trials
Observational studies: cohort studies, case-control studies, case-series
Is this diagnostic test accurate?
Cross-sectional or case-control study designs
What will happen if we do not add a therapy?
Cohort studies with incident cases
What is the frequency of a condition or health-related problem in the population? How many people are affected?
Evidence-Based Healthcare and Systematic Reviews
As stated above, the scientific knowledge is always evolving. Our comprehension about the effects of the healthcare interventions, being it a treatment or diagnostic test, is continuing evolving too. However, this oath is frequently ignored and a static piece of information, such as that provided by a product manufacturer or by one published paper, is continuously used to guide decisions.
As a case example, the development of a drug intervention starts with the chemical design and then follows a series of preclinical tests consisted of laboratory and animal experiments. If the drug under investigation succeeds in the preclinical tests, further researches are conducted in human under controlled situations. In this process, the laboratory tests performed in the pre-clinical phase are useful to test the intervention in disease models—animals. The potential efficacy and relativeness safety of the new treatment need to be evaluated in real patients, in a clinical phase constituted of a series of studies named clinical trials. Together, the pre-clinical and clinical data provide relevant answers related to a medical intervention might include a drug, a medical device, or a screening method. Nevertheless, our knowledge about these healthcare interventions is limited at the time they are granted marketing approval. For instance, at the time a drug is approved to be used in real-life situations, only several hundreds to about 3000 volunteers who have the disease to be treated are expected to have been tested [20, 21]. In Europe, from 2000 to 2010, the median total number of patients studied before a drug approval was 1708 .
The above numbers demonstrate that the accumulated data related to any intervention is very limited at the time marketing approval is granted. Moreover, to be approved in the United States, there must be adequate data from just two clinical trials . After marketing approval is granted, additional studies and reanalysis of the available published and non-published results may depict more details about the efficacy and harms of the intervention. In addition, the clinical phase of tests continues with observational studies intended to accumulate data on harmful effects. This is essential since clinical trials can only detect frequently and predict harm outcomes, and a complete investigation of all types of harmful effects induced by a healthcare intervention requires the assessment of mainly non-randomised studies . It may now be clear why “no study, whatever the type, should be interpreted in isolation” , and why the evidence gathered by all the studies related to a given intervention should be taken collectively to inform healthcare decisions . The research method more capable of accomplishing such a critical overview of the evidence is the so-called systematic review.
A systematic review may be understood as a research undertaken with previous completed studies. Therefore, a systematic review uses an explicit method to collate the primary studies that answer a specific research question . A qualified systematic review may also include a quality assessment of the studies collated, i.e., a judgment of the risk of bias of each included study. A systematic review may or may not include a meta-analysis, which is a statistical method to combine the results of independent studies . Consider the treatment of intermittent claudication discussed above. Omega-3 fatty acid  has been studied to improve the pain-free walking distance and the maximum walking distance achieved by patients after treatment. Since this is a treatment question, if we search for clinical trials investigating the problem we could find one study published in 2005, which appears to demonstrate that omega-3 is effective to improve the pain-free walking distance and probably an option to also improve the maximum walking distance of individuals with intermittent claudication . However, another trial failed to support the hypothesis that omega-3 was effective to improve either the pain-free walking distance or the maximum walking distance . The combined effect of these trials shows that there is no evidence that omega-3 consistently improves clinical outcomes of patients with intermittent claudication (Fig. 1.1) . If we have not appraised the combined effect of both of these trials in a systematic review, we would be taking decisions informed by biased evidence.
Statistical illustration of two trials demonstrating opposite effects on the pain-free walking distance of omega-3 fatty acid in the treatment of intermittent claudication . The figure presents a graphic name to forest plot. In a forest plot, the point estimate of the result of the individual studies are shown as squares centred, and the confidence interval is represented by a line crossing through the square. The combined estimate from the meta-analysis and its confidence interval is represented by a diamond at the bottom of the graphic . The parallel line crossing the graphic where the mean difference is equal to zero accounts for a result that is equal in the intervention and comparison group. Reprinted with permission from: “Campbell A, Price J, Hiatt WR. Omega-3 fatty acids for intermittent claudication. Cochrane Database Syst Rev. 2013;7:CD003833”.
Harms , Not Just Efficacy
Every healthcare intervention will carry a risk—great or small—of inducing harm effects causing injury or disability to patients. Usually, optimistic misconceptions about harms induced by healthcare interventions result in researchers and clinicians inadvertently not taking data on harm in consideration when reporting a study or making decisions in healthcare. Since research reports frequently fail to detail data on harmful effects, the intervention may be erroneously declared safe [32–37]. However, “absence of evidence of harm should not be construed as evidence of absence of harm” , and at most we will have data on a relativeness safety of an intervention in these situations. Another issue complicating the evaluation of harms is the several terms used to describe all the harmful events that can be associated to healthcare intervention [23, 38] (Table 1.2). Furthermore, some of the harm effects may be recognised by the healthcare professionals as mild and non-relevant events. Even mild or moderate harmful effects could be of major significance for a treated patient, resulting in poor treatment adherence and in the use of additional medications to treat the harm effect. Drug-induced acute or chronic diarrhoea, which is one of the harms induced by omega-3 and cilostazol [10, 28], can be severe and poorly tolerated . A headache, another harm effect reported by patients treated with cilostazol , imposes a recognisable burden on sufferers, including substantial personal suffering, impaired quality of life, and financial cost [40, 41].
Terminology of harmful effects
An unfavourable outcome that occurs during or after the use of a drug or another intervention but is not necessarily caused by it
An adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility
Adverse drug reaction
An adverse effect specific to a drug
This is an old term and should no longer be used because it underestimates the importance of harms associated to healthcare interventions
Any unintended effect, adverse or beneficial, of a drug that occurs at doses normally used for treatment
The totality of all possible adverse consequences of an intervention
The inconsistent report of the frequency of events related to harm effects is another challenge when assessing data on harmful effects. A biased report may often describe a harm effect as rare when it is, in reality, a common effect. The Uppsala Monitoring Centre (UMC), the international drug monitoring programme of the World Health Organization (WHO) , suggests a standard classification for the frequency of harmful effects induced by healthcare interventions (Table 1.3) .