Healthy Skepticism Library item: 13532

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: Journal Article

Editorial .
Stopping trials early for benefit: too good to be true
The Lancet 2008 Apr 19; 371:(9621):1310
http://www.thelancet.com/journals/lancet/article/PIIS0140673608605693/fulltext

Full text:

When a randomised controlled trial (RCT) demonstrates dramatic superiority at interim analysis, stopping that trial early could save money and lives by hastening the availability of an effective intervention. In a study published on April 9 in Annals of Oncology, Francesco Trotta and colleagues examined the strength of data from trials for anticancer drugs that had been stopped early for benefit. Their findings question the robustness of short-term data. Of the 25 trials, six had no data and safety monitoring board (DSMB) and five had enrolled less than 40% of the sample size. Even so, 11 were used to support licensing applications on the basis of what could have been exaggerated chance events.

Controversy about the scientific credibility of RCTs stopped early for benefit is not new. In 2005, Victor Montori and colleagues published a systematic review of 143 such trials (55 of which had been published in the New England Journal of Medicine). Half the studies were halted after the first interim analysis, many driven by composite endpoints. Reporting was poor: only eight RCTs followed CONSORT guidelines. The median risk ratio was 0·53 for the intervention after 66 events in 64% of the sample size during 13 months’ follow-up. Despite the fact that smaller data sets overestimate putative effect sizes, sensational positive results receive disproportionate attention and can bias practice, guidelines, and meta-analyses.

The sense of urgency to stop a trial early could be misleading, since Trotta’s group found a 2-year delay between stopping and publication. To stop a trial early should therefore require proof beyond reasonable doubt that equipoise no longer exists. DSMBs must balance the decision to stop, which favours immediate stakeholders (participants, investigators, sponsors, manufacturers, patients’ advocates, and editors), with continuing the study to obtain more accurate estimates of not only effectiveness, but also of longer-term safety. In judging whether or not to stop a trial early for benefit, the plausibility of the findings and their clinical significance are as important as statistical boundaries. The same principles should guide the application of such findings to policy and practice.