Healthy Skepticism Library item: 18358

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: Journal Article

Clarke M, Hopewell S, Chalmers I
Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting
Lancet 2010 July 3; 376:(9734):20-1
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(10)61045-8/fulltext

Abstract:

If a new clinical trial is to be justifiable both scientifically and ethically it should be designed in the light of an assessment of relevant previous research, ideally a systematic review.1 When its findings are reported, these should be set in the context of updated reviews of other, similar research.2
In 1997,3 2001,4 and 2005,5 we assessed reports of randomised trials published in the month of May in five medical journals: Annals of Internal Medicine, BMJ, JAMA, The Lancet, and the New England Journal of Medicine. Only a small proportion of trial reports provided sufficient information to assess the contribution of the new results to the totality of the available evidence. We repeated our study in May, 2009. As previously, we assessed the Discussion sections of the trial reports and, as in 2005, we also investigated the extent to which reports referred to systematic reviews used in the design of the new research in their Introduction sections.
Our criteria for including a report as a “trial” were as before: it was a randomised or quasi-randomised trial; it was mainly concerned with the outcomes studied in the trial; and it was published as a full report or paper in one of the five journals in May, 2009. One of us (SH) searched the relevant issues of the journals to identify eligible reports. The Introduction and Discussion sections of each eligible report were then assessed independently by at least two of us to decide whether they referred to a systematic review or included an updated systematic review. If a trial claimed to be the only trial of a topic, we searched for trials in the Cochrane Central Register of Controlled Trials (CENTRAL), which might be considered for inclusion in a systematic review of the topic. We resolved disagreements by discussion between all three of us.
28 reports of randomised trials were identified. Systematic reviews were referred to in the Introduction section of 11 reports. In five of the 28 reports, the authors claimed that their study was the first to have addressed the question concerned. Some of these five reports cited other systematic reviews as proof of this claim,6, 7 or gave details of the search they had done.8 Our searches of CENTRAL did not identify apparently similar trials for any of the five claiming to be the “first trial”.
One of the 24 reports of trials that did not claim to be the first trial placed the results of the new trial in the context of an updated systematic review of other research in the Discussion section.9 Reference was made to relevant systematic reviews in ten other reports, without any integration of the results of the new trials into an update of these reviews. In the remaining 13 reports, there was no evidence that any systematic attempt had been made in the Discussion section to set the new results in the context of previous trials (table). …