Healthy Skepticism Library item: 14499

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Rawlins criticises traditional methods of assessing evidence
PM Live 2008 Oct 15
http://www.pmlive.com/pharm_market/news.cfm?showArticle=1&ArticleID=7115

Full text:

In this year’s Harveian Oration, delivered before the Fellows of The Royal College of Physicians of London on Thursday October 16, 2008, Professor Sir Michael Rawlins, Chair of the National Institute for Health and Clinical Excellence (NICE), argues that a new approach to analysing clinical evidence is required.

According to Sir Michael’s oration, randomised controlled trials (RCTs) have been put on an undeserved pedestal. Despite being billed as “the gold standard” for demonstrating (or refuting) the benefits of a particular intervention, Sir Michael claims RCTs have important limitations and imperfections, including inappropriateness, utility of the null hypothesis, theories of probability, generalisability of the results and resource implications. He states that their appearance at the top of “hierarchies” of evidence is inapt; and hierarchies, themselves, are illusory tools for assessing evidence.

Sir Michael recommends that RCTs be replaced by a diversity of approaches that involve analysing the totality of the evidence-base. In his opinion, observational studies are useful and, with care in the interpretation of the results, can provide an important source of evidence about both the benefits and harms of therapeutic interventions. In particular these include historical controlled trials and case-control studies, but other forms of observational data can also reveal important issues.

Sir Michael rejects the trend towards grading various kinds of clinical trials and studies on scales of merit, which he says have come to dominate the development of some aspects of clinical decision-making.

Arguments about the relative importance of different kinds of evidence are an unnecessary distraction, he claims. What is needed instead is for “investigators to continue to develop and improve their methodologies; for decision makers to avoid adopting entrenched positions about the nature of evidence; and for both to accept that the interpretation of evidence requires judgement”.

In conclusion, Sir Michael states: “I am aware that those who develop and use hierarchies of evidence are attempting to replace judgements with what, in their eyes, is a more reliable and robust approach to assessing evidence. All my experience tells me they are wrong. It is scientific judgement – conditioned, of course, by the totality of the available evidence – that lies at the heart of making decisions about the benefits and harms of therapeutic interventions”.