Healthy Skepticism Library item: 11808

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Taylor L.
NICE failing to use experts and understand evidence, MPs told
Pharma Times 2007 Oct 19
http://www.pharmatimes.com/Article/07-10-19/NICE_failing_to_use_experts_and_understand_evidence_MPs_told.aspx

Full text:

Clinical recommendations produced by the UK National Institute for Health and Clinical Excellence (NICE) have on occasion been “fundamentally flawed,” due to misrepresentation of the evidence base, and some have also gone against internationally-accepted, evidence-base best practice, UK Members of Parliament (MPs) have been told.

One problem is that members of the independent Guideline Development Groups which are set up under the NICE process to review the data and provide guidance are not experts in the area under study, nor is the Group’s chairman required to be an expert, Beverley Hunt, medical director of the thrombosis charity Lifeblood, has told the House of Commons Health Select Committee.

While it is laudable to exclude those with a possible conflict of interests from the Group, it also may result in members who lack a detailed understanding of the scientific data which they are reviewing and who are unable to recognise the quality of the studies, she warned.

Dr Hunt recommended that the recruitment criteria for membership of the Groups should be revised to ensure that they include experts in the clinical area being studied, on the proviso that they declare all conflicts of interests. Only those with “substantial interests” – eg, more than 10% of their gross salary or grant income over £100,000 coming from “interested parties” – should be excluded, she suggested, and pointed out to the panel that “you can’t be an expert if you don’t have dealings with pharmaceutical companies.”

Moreover, Dr Hunt noted that, while experts are consulted to give evidence to the committee, there are no predefined rules as to what NICE considers “an expert,” and proposed that it should use a simple but formal assessment tool to determine expertise, for example evidence of substantial clinical publications in the area being reviewed. Because of the length of time which the NICE appraisal process takes, Dr Hunt felt it would be better for the experts to be members of the Guideline Development Groups, but she also accepted the suggestion from Select Committee member Charlotte Atkins (Labour, Staffordshire Moorlands) that they could serve as professional advisers throughout the assessment process, in the way that such specialists are present to advise the Commons panel throughout its investigations.

NICE also needs to revise the criteria which it uses to gather its data and move away from only looking at randomised controlled trials, Dr Hunt went on. Currently, many large studies are being ignored, even though they are worthwhile – albeit imperfect – yet very small randomised trials are considered useful despite being significantly limited in terms of their statistical power. Moreover, there is a tendency to used mixed meta-analysis – lumping together a large number of studies which experts would argue were completely different – which can lead misleading conclusions, she warned.

Dr Hunt’s comments specifically referred to the recent draft guideline on prevention of pulmonary embolism in high-risk surgical patients, published in October 2006, but she added that similar problems have also been experienced with draft documents dealing in the areas of hypertension and hyperlipidaemia, which have needed dramatic revision at a late stage before final publication. More recently, concerns have been raised, for similar reasons, about the post-myocardial infarction guideline process, she told the MPs.

Roger Atkins, the British Orthopaedic Association’s (BOA) lead councillor for NICE, told the Committee that the orthopaedic community was not adequately consulted during production of NICE’s guidance on the prevention of venous thromboembolism (VTE) in patients undergoing surgery. The “very didactic” recommendations contained in the guideline, which was published in April, are not those recommended by orthopaedic VTE experts and they also differ from the “seminal” recommendations of the American Academy of Orthopaedic Surgeons, he said.

Prof Atkins was highly critical of the fact that none of the orthopaedic experts who have “a lifelong interest in the area” were invited to give advice on the guideline’s development, and also that the BOA’s lengthy commentary on the draft received no response and scant notice in the final version.

There appears to be no ready mechanism for NICE to review or vary its guidance, once it is published, in light of the clinicians’ very serious concerns, commented Prof Atkins. He also warned the panel that implementation of the VTE guidance will be expensive and “may well be associated with significant complications.”