After reading the meta-analysis on whole-grain intake and coronary heart disease (CHD) risk by Tang et al in the March 2015 issue of the American Journal of Cardiology , I considered the need to provide some constructive feedback. Systematic reviews and meta-analyses are regarded as being crucial to evidence-based practice, and information on protective factors for CHD is highly relevant, given that CHD is the leading cause of death in the United States. However, because of several major flaws in its execution, I have serious reservations about the findings of this review.

First, the investigators failed to synthesize all available research on the topic into their review. For example, they only used 2 electronic search databases, when additional studies could have been identified through other sources such as the Cochrane Library or EMBASE. Also, the authors only included observational studies (namely prospective cohort and case-control) into the review, providing no explanation as to why they did not include more rigorous designs such as randomized controlled trials. When I performed a search using the authors’ search string on PubMed, one of the databases they used, I managed to find at least 4 trials that may have been eligible for inclusion into this review (citations available on request).

Second, the authors did not seem have assessed the scientific quality of the included studies, negatively affecting the transparency of the review process. Thus, readers cannot properly assess its quality as a comprehensive review of the literature or assess the internal validity of the meta-analysis. Given that a number of reporting guidelines for reviews have been produced, these issues are almost inexcusable. In fact, when I applied the checklist for Meta-analysis of Observational Studies in Epidemiology, I found that 13 of the list’s 33 elements were not reported adequately in this article (checklist available on request).

Despite being considered some of the strongest forms of medical evidence, like any other type of study, meta-analyses greatly vary in quality. Although the investigators successfully addressed issues of heterogeneity and excessive influence from a single study, unfortunately, the review’s blatant methodologic weaknesses prevent it from being an effective tool for assisting clinical decision making. I hope that for future analyses, the investigators consider these points and the reporting protocols found in the Meta-analysis of Observational Studies in Epidemiology and other valuable guidance statements to improve the quality of their work.