Healthy Skepticism Library item: 5480

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Chalmers I.
From optimism to disillusion about commitment to transparency in the medico-industrial complex
Journal of the Royal Society of Medicine 2006 Jul 1499:(7):337-341
http://www.jrsm.org/cgi/content/full/99/7/337

Notes:

Ralph Faggotter’s comments:

‘Underreporting research is scientific misconduct’

It is equally encumbent upon researchers to report ‘negative’ outcomes of research as it is to report the ‘positive’ outcomes.

How else can we accumulate knowledge?

But this is not what happens in the real world where negative findings often go reported- especially when large amounts of money are at stake!

The methodical accumulation of accurate scientific knowledge frequently runs a poor second to commercial imperatives.

Full text:

From optimism to disillusion about commitment to
transparency in the medico-industrial complex
Iain Chalmers
J R Soc Med 2006;99:337–341
In the mid-1970s, I was involved with others in trying to
assemble evidence about the effects of care during
pregnancy and childbirth. We started by searching Medline
to identify reports of controlled trials that we thought might
be sufficiently reliable to be taken into account in our
analyses; and then searched about 70 journals by hand, page
by page, back to the issues published in 1950. We found
about 3500 reports of controlled trials and, with help from
the World Health Organization, we published the
references to these in a classified bibliography in the mid-
1980s.1
But then we thought, ‘What about trials that don’t get
published? If we don’t take those into account, we might
reach biased conclusions’. We knew of anecdotes about
researchers who had had disappointing results of trials, and
had therefore never made the results public. Furthermore,
in an important analysis of controlled trials of treatments
for cancer of the ovary published in 1986, an Australian
oncologist, John Simes, had shown how failure to publish
clinical trials could lead to misleading inferences about the
effects of treatments.2 So we wrote to 42 000 obstetricians,
paediatricians and other clinicians around the world to try
to flush out information about studies that had been done
but had never been reported. It was an almost complete
waste of time; we published our experience to warn others
to think carefully before using the retrospective survey
approach we had used in our attempt to deal with
publication bias.3
Like John Simes before us, we concluded that a
prerequisite for tackling the problem of biased underreporting
of research involved registering controlled trials
publicly, at inception, before their results could influence
whether or not the world would come to know about them.
In 1990, based on a paper I had presented at the 1st
Congress on Peer Review in Biomedical Publishing, I
published an article with the deliberately provocative title
‘Underreporting research is scientific misconduct’. I
suggested that failing to report well-conducted clinical
trials was not only scientific misconduct, but also unethical;
it broke an implicit contract with the patients who had
participated in clinical trials. I reiterated the call for
registration of trials at inception.4
As it happens, all the examples of biased underreporting
of controlled trials I gave in that paper referred to
non-commercial trials. The drug industry is not very
interested in perinatal healthcare, the field in which I was
researching at that time, because it does not offer very rich
pickings. Ten years before my article was published,
however, Elina Hemminki had shown that biased underpublication
of studies funded by industry might be a
particular cause for concern.5 She compared the study
reports submitted by pharmaceutical companies to drug
licensing authorities with subsequently published reports of
the same studies. From this she found that studies in which
the pre-licensing records showed that researchers had
looked for adverse effects were less likely to be published
than studies in which adverse effects had not been sought.
For the reasons already mentioned, I had had little
contact with the pharmaceutical industry while I was a
perinatal researcher. In 1992, however, I moved to the UK
Cochrane Centre. As the Centre and then the Cochrane
Collaboration began to become known to people in
industry, representatives of the Association of the British
Pharmaceutical Industry asked to meet with me. A
delegation came to the UK Cochrane Centre, led by Frank
Wells, the medical director of the ABPI. Our discussion
lasted for about 3 h, during which I emphasized the
openness with which the Cochrane Collaboration did
its work: protocols for Cochrane reviews and
completed reviews were published in electronic form,
and could be criticized publicly, and readily amended if
necessary.
After I had explained this, I felt it reasonable to suggest
to my ABPI visitors that industry should be more open
about its clinical research, because biased under-reporting
could harm patients. Encouragingly, Frank Wells subsequently
wrote in the preface of a book he had co-authored
on fraud and misconduct in medical research: ‘Underreporting
of research is (another) form of misconduct, given
that this can lead to seriously misleading recommendations
for clinical practice and for new research’.6
The same year that this book was published, Mike
Wallace, another member of the ABPI delegation and at
that time the chief executive of Schering Healthcare Ltd
ESSAYS
337
J O U R N A L O F T H E R O Y A L S O C I E T Y O F M E D I C I N E V o l u m e 9 9 J u l y 2 0 0 6
Editor, James Lind Library
E-mail: ichalmers@jameslindlibrary.org
(the British subsidiary of Schering AG), told me that he
realized that biased under-reporting of research was
ethically and scientifically indefensible. He felt that unless
the pharmaceutical industry started to take an initiative to
confront this issue voluntarily it would end up being forced
to do so—probably in ways that it would rather not be.
Mike Wallace said that he had decided to provide
information about all his company’s controlled trials for
publication in the Cochrane Controlled Trials Register. I
know that he was reprimanded for breaking ranks in this way,
in particular, by colleagues from other member companies of
the ABPI. I am glad to be able to pay tribute to a senior nonmedical
executive in the industry who stepped out of line to
do something which he thought was morally correct.
Mike Wallace was not the only person within the British
pharmaceutical industry taking steps towards greater
openness, however. In particular, David Jackson, medical
director of the British holding company of GlaxoWellcome,
had persuaded his colleagues in the international parent
company that the company should adopt a more open
policy. The new policy was announced at the end 1997, and
I was pleased to be quoted in the press release welcoming
it. The following year, the chief executive of the company,
Richard Sykes, wrote a landmark editorial in the BMJ
entitled ‘Being a modern pharmaceutical company involves
making information available on clinical trials programmes’.
He made clear that one reason for Glaxo Wellcome’s new
disclosure policy was ‘. . . to help those undertaking
systematic reviews of clinical data and to help reduce the
impact of publication bias’.7
GlaxoWellcome implemented the trials registration
policy it had announced, and two of the people who were
involved—Trevor Gibbs and Elizabeth Wager—later
published an account of the challenges this had presented.8
Elizabeth Wager was head of the writers’ group within the
company and she convened a group of her colleagues in
other companies to agree and then publish guidelines for good
publication practice. They noted that ‘. . . the aim of the
guidelines is to ensure that clinical trials sponsored by
pharmaceutical companies are published in a responsible and
ethical manner. The guidelines cover companies’ responsibilities
to endeavour to publish results of all studies’.9,10
Things were stirring among pharmaceutical physicians
more widely. In 1998 the Ethics Committee of the Faculty
of Pharmaceutical Medicine declared that:
‘Pharmaceutical physicians have a particular ethical responsibility
to ensure that the evidence on which doctors should make
their prescribing decisions is freely available . . . the outcome of
all clinical trials on a medicine should be reported’.11
Two years later they reiterated the ethical principle:
‘Studies are performed to increase knowledge in some way,
and this knowledge should be shared with the wider world.
Study findings should be communicated, whatever the
outcome, for the benefit of the community at large’.12
Presumably because the ABPI felt that it could not really
sit by while one of its major members, GlaxoWellcome,
and the Ethics Committee of the Faculty of Pharmaceutical
Medicine were showing a moral lead in this way, the
Association decided to commend Glaxo Wellcome’s policy
to other member companies. I was on the ABPI side of the
table at the press conference announcing this policy, and I
welcomed it wholeheartedly. In brief, I had become
optimistic that the industry really was beginning to address
the problems that I had urged should be addressed at our
meeting eight years previously. Although the response was
not nearly as good as the ABPI had hoped, AstraZeneca,
Aventis, MSD, Novartis, Roche, Schering Healthcare and
Wyeth did begin registering retrospectively those of their
trials that had involved UK patients.
However, these hopeful signs were not sustained during
subsequent developments. In the same year that the ABPI
announced that it was leading the world in registering
clinical trials, Glaxo Wellcome merged with Smith Kline
Beecham. Initially, my optimism that the new company
would maintain Glaxo Wellcome’s policies on trial
registration seemed justified: the new company’s logo had
replaced the Glaxo Wellcome logo on the trials register. I
wrote to the chief executive of the merged company,
Jean-Paul Garnier, to express my pleasure at this. I also
mentioned that I was delighted that Elizabeth Wager and
her six colleagues in the former. Glaxo Wellcome writing
department, and their association with the Good Publication
Practice for Pharmaceutical Companies Guidelines,
continued to be supported by the new company. I never
received a reply to my letter; the company abandoned the
trial registration process and abolished its publications
departments in the USA and the UK, headed by Elizabeth
Wager and Elizabeth Field, respectively—both co-authors
of the Good Publication Practice Guidelines for Pharmaceutical
Companies.9,10
It was round about that time that increasing amounts of
empirical evidence began to confirm doubts about the
trustworthiness of research sponsored by industry. As I have
noted already, Elina Hemminki had produced evidence of
based under-reporting of industry research 20 years earlier:
reports of studies submitted in support of new drug licences
in which adverse effects had been sought were less likely
subsequently to be published.5 A study using similar
methods was reported in 2003 by Melander and his
colleagues.13 Their paper entitled ‘Evidence b(i)ased
medicine—selective reporting from studies sponsored by
pharmaceutical industry’ demonstrated result-dependent
over-reporting and under-reporting of industry studies of new
338 drugs. They concluded that any attempt to develop treatment
J O U R N A L O F T H E R O Y A L S O C I E T Y O F M E D I C I N E V o l u m e 9 9 J u l y 2 0 0 6
recommendations using analyses based only on publicly
available data were likely to be based on biased evidence.
Other evidence reported over the past 5 years has
demonstrated associations between industry-sponsorship
and research results favouring products made by the
companies funding the research. As well as biased underreporting
of research, the associations observed may also
reflect inappropriate comparison of new products with
comparators of existing products in doses too low to be
effective or higher than necessary, with consequent higher
incidence of adverse effects.14–17 Neither of these possible
explanations is morally or scientifically defensible. In
addition, there is evidence of more subtle forms of industry
bias in the way that data are interpreted,18,19 for example,
by putting a biased spin on the evidence. The title of a
recent report20 says it all: ‘Why olanzapine beats
risperidone, risperidone beats quetiapine, and quetiapine
beats olanzapine: an exploratory analysis of head to head
comparison studies of second generation anti-psychotics’.
One of the five key points emerging from the evidence
presented at the 5th Congress on Peer Review in
Biomedical Publishing was that ‘. . . the drug industry is
successfully skewing the literature in its favour’.21
By now the public was starting to get interested in these
matters. I was asked to write an article about underreporting
of research for the popular weekly journal New
Scientist.22 In fact, I had co-authored a piece on the topic in
the same journal 8 years previously.23 But now, something
happened which made a real difference to the likelihood of
the message about publication bias being taken seriously.
Eliot Spitzer, the attorney general for New York State, took
GlaxoSmithKline to court for withholding information
about important possible adverse effects of drugs taken by
children for depression.
The company settled out of court for a couple of million
dollars, but the charge laid against it had the effect of
increasing the pressure to deal with publication bias. It
provoked the International Committee of Medical Journal
Editors to do what the Committee should have done years
earlier. It announced that the authors of any reports of
clinical trials that had begun recruitment after the middle of
2005 would have to confirm that the studies had been
registered publicly, at inception.
Only then did the drug industry start to try to limit the
damage caused by its failure to follow the lead given by
Schering Healthcare and Glaxo Wellcome nearly a decade
earlier. In a press release issued in June 2004,
GlaxoSmithKline announced that it would make the results
of all its clinical trials publicly available. The company did
not explain how, given that its trials register had been
allowed to fall into disrepair, the public would know
whether all results were being made available. A BMJ
editorialist commented thus:
‘Last month GlaxoSmithKline announced that it would publish
summaries of all its clinical trials of a new product once it had
been launched. The decision followed news of a lawsuit brought
by New York State alleging that the company had concealed the
results of paroxetine because they might have spoiled marketing
plans. GSK said it had been considering the move for some
months. A similar sounding policy was announced by Glaxo
Wellcome in 1998, but seems to have been quietly abandoned in
2000 after the merger with SmithKlineBeecham.’24
Industry’s attempts at damage limitation continue. Speaking
on the BBC Radio 4 programme In Business on 24 June
2005, Jean-Pierre Garnier said: ‘We have responsibilities
beyond those to our shareholders’, and ‘. . . we must be
completely transparent with the public about what we do’.
The gap between this statement and an analysis of the
records submitted by his company (and other pharmaceutical
companies) to the National Institutes of Health trials
register25 suggests that GlaxoSmithKline’s understanding of
what transparency implies may leave many people
disappointed and cynical about the company’s expressed
commitment to greater openness.
If GlaxoSmithKline really is committed to the kind of
openness the public has come to realise is needed, one
might have expected it to have endorsed the Good
Publication Practice for Pharmaceutical Companies Guidelines
developed by its dismissed former employees and
colleagues in other companies.9 It has not. Indeed, at the
time of writing, only six pharmaceutical companies have
endorsed the guidelines—Aventis, Amgen, LEO Pharma,
Otsuka, Serono and 3M Pharmaceuticals.10
The public is beginning to demand stronger sanctions.
During 2004 and 2005, the Health Committee of the House
of Commons held an enquiry into the influence of the
pharmaceutical industry. The Committee drew attention to
the problem of biased under-reporting of research and
recommended registration of trials at inception.26 Richard
Sykes, previously chief executive of Glaxo Wellcome, now
rector of Imperial College London, gave oral evidence to
the Committee which clearly impressed the members. In its
report, the Committee’s recommendations began by noting
Richard Sykes’ view that:
‘Today the industry has got a very bad name. That is very
unfortunate for an industry that we should look up to and
believe in, and that we should be supporting. I think there have
to be some big changes.’
On 6 January 2005, the industry announced a global
commitment to clinical trial registration and publication.
Whether this commitment amounts to the ‘big changes’
that Richard Sykes judges are needed is an open question.
First, the commitment applies to only a tiny proportion of 339
J O U R N A L O F T H E R O Y A L S O C I E T Y O F M E D I C I N E V o l u m e 9 9 J u l y 2 0 0 6
the clinical trials which should be informing prescribing
practices because it is not being applied to trials conducted
in the past. Second, there is a continuing flow of empirical
evidence of biased reporting and interpretation of industry
trials. My judgement is, therefore, that industry’s actions
are simply too little and too late to deal with its current
reputation for being less than honest with its research—just
as Mike Wallace feared might be the case in his
conversations with me a decade ago.
Biased under-reporting of research harms and sometimes
kills patients, quite apart from the waste of resources
that results from this form of scientific and ethical
misconduct. In 2004, a former editor of the New England
Journal of Medicine published a book entitled The Truth About
The Drug Companies: How They Deceive Us and What To Do
About It.27. How can we expect the public to trust the
pharmaceutical industry and clinical researchers who work
with it while we acquiesce in this state of affairs?
Jan Vandenbroucke has noted how science as an errordetecting
process simply ceases to exist in these
circumstances:
‘In all scientific debates all sides always have their own biases:
we have no other way to look at data but to interpret them.
However, in usual clinical or epidemiologic research, studies are
repeated by others, in different settings and by different means,
looking for biases, flaws, and ways of remedying them, endlessly
arguing whether the biases are remedied or not. That is the
essence of open scientific debate and criticism, which is the only
guarantee for progress. That is no longer possible with
pharmaceutical products because the monopoly of the
pharmaceutical industry of studies of its own products leads to
persistently one-sided studies that can no longer be questioned by
studies from other sides. Moreover, the one-sidedness cannot be seen
from the public record, that is the published papers. Without the
possibility of open debate, science simply ceases to exist’.28
Careful thought needs to be given to this analysis by
governments, public and charitable organizations, and
individuals who promote research collaboration with
industry while remaining silent about the unethical and
unscientific behaviours alluded to above. As I have made
clear, biased reporting and lack of transparency is not
limited to commercially-sponsored research; but the
empirical evidence makes clear that it is a particularly
noticeable problem in that sphere. Those who collaborate
with industry need to be clear that their acquiescence in
these forms of scientific misconduct inevitably casts doubt
on their integrity, as reflected in the title of a book by
another former editor of the New England Journal of
Medicine—On The Take: How Medicine’s Complicity With Big
Business Can Endanger Your Health.29 The responsibilities of
doctors should be unambiguously to their patients. Yet
some clinician researchers (with plenty of encouragement
from government and the institutions with which they are
associated) appear to have become so seduced by the
financial rewards resulting from collusion with industry’s
agenda and practices that they have forgotten this most
fundamental of their professional duties. Indeed, that is the
most depressing cause of my optimism of a decade ago
becoming disillusionment today.
A few weeks before finalizing a draft of this essay for
submission to the Journal of the Royal Society of Medicine I was
invited by Richard Tiner, medical director of the ABPI, to
address a meeting of 60 or so members of medical
departments of member companies on the topic of ‘clinical
trial transparency’. In response I suggested that, rather than
give a talk, it might be a more useful to pre-circulate the
above text so that it could be discussed at the meeting, and
Dr Tiner accepted this proposal.
It came as a bit of a surprise to me that there was no
serious challenge to the facts I had assembled to explain why
my earlier optimism had turned to disillusion. This may
have been partly because it was clear that I have tried to
work with industry over the past decade to increase
transparency, and to commend publicly the positive steps
that some individuals, companies and the ABPI had taken in
that direction. Instead of giving me the rough ride that I had
been expecting, participants at the meeting asked me for
suggestions about what could be done to deal with the bad
reputation to which Richard Sykes had referred in his evidence
to the Health Committee of the House of Commons.
Although I made one or two off the cuff suggestions at the
meeting, I realized that I needed to add to this essay some
suggestions for steps that might go some way to increasing
confidence in the scientific integrity of the industry. So, in
conclusion, here are three practical suggestions:
1 All pharmaceutical companies should join Aventis,
Amgen, LEO Pharma, Otsuka, Serono and 3M Pharmaceuticals
in publicly endorsing the Good Publication
Practice Guidelines for Pharmaceutical Companies9,10
2 Industry as a whole should put in place mechanisms for
promoting and monitoring adherence to these guidelines,
thus making clear to the public that it regards
under-reporting of research as just as serious a form of
scientific misconduct as fabrication of data
3 Industry should voluntarily take steps that go beyond
the minimum currently being required for clinical trial
registration,30 for example, by publishing full protocols
at the inception of all randomized trials.
I hope that I may be able to write another essay in 5 years’
time entitled ‘From disillusion to optimism about the
scientific integrity of the pharmaceutical industry and the
340 people collaborating with it.’
J O U R N A L O F T H E R O Y A L S O C I E T Y O F M E D I C I N E V o l u m e 9 9 J u l y 2 0 0 6
Acknowledgments I am grateful to Richard Sykes,
Richard Tiner, Elizabeth Wager, Mike Wallace, Frank
Wells, Jan Vandenbroucke, and participants at the ABPI
meeting held at BMA House on 28 February 2006, for
helping me to ensure that this account is accurate.
Competing interests None declared.
Note Listen to an audio interview with Iain Chalmers on
[http://www.jrsm.org]
REFERENCES
1 National Perinatal Epidemiology Unit. A Classified Bibliography of
Controlled Trials In Perinatal Medicine 1940–1984. Oxford: Oxford
University Press (for the World Health Organization), 1985
2 Simes RJ. Publication bias: the case for an international registry of
clinical trials. J Clin Oncology 1986;4:1529–41
3 Hetherington J, Dickersin K, Chalmers I, Meinert CL. Retrospective and
prospective identification of unpublished controlled trials: lessons from a
survey of obstetricians and pediatricians. Pediatrics 1989;84:374–80
4 Chalmers I. Under-reporting research is scientific misconduct. JAMA
1990;263:1405–8
5 Hemminki E. Study of information submitted by drug companies to
licensing authorities. BMJ 1980;280:833–6
6 Lock S, Wells F. Preface to the second edition. In: Lock S, Wells F,
eds. Fraud and Misconduct In Medical Research. London: BMJ Publishing
Group, 1996:xi–xii
7 Sykes R. Being a modern pharmaceutical company. BMJ 1998;
317:1172
8 Gibbs T, Wager E. Realities of trial registration: the Glaxo Wellcome
experience. Int J Pharmaceut Med 2000;14:203–5
9 Wager E, Field EA, Grossman L. Good Publication Practice for
pharmaceutical companies. Curr Med Res Opinion 2003;19:149–54
10 [http://www.gpp-guidelines.org]
11 Faculty of Pharmaceutical Medicine. Ethical Issues Working Group.
Ethics in pharmaceutical medicine. Int J Pharmaceut Med 1998;12:193–8
12 Wells F, Lunnon MW. First Report of the Ethics Sub-Group. Society
of Pharmaceutical Medicine. Int J Pharmaceut Med 2000;14:58–64
13 Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence
b(i)ased medicine—selective reporting from studies sponsored by
pharmaceutical industry: review of studies in new drug applications.
BMJ 2003;326:1171–3
14 Djulbegovic B, Lacevic M, Cantor A, et al. The uncertainty principle
and industry-sponsored research. Lancet 2000;356:635–8
15 Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry
sponsorship and research outcome and quality: systematic review. BMJ
2003;326:1167–70
16 Bhandari M, Busse JW, Jackowski D, et al. Association between
industry funding and statistically significant pro-industry findings in
medical and surgical randomized trials. Can Med Assoc J 2004;170:477–
80
17 Gardner W, Lidz CW. Failures to publish pharmaceutical clinical
trials. Proceedings of The 5th International Congress On Peer Review and
Biomedical Publication, Chicago, September 2005.
18 Yank V, Rennie D, Bero L. Are authors’ financial ties with
pharmaceutical companies associated with positive results or
conclusions in metaanalyses on antihypertensive medications?
Proceedings of The 5th International Congress On Peer Review and
Biomedical Publication, Chicago, September 2005.
19 Jørgensen AW, Gøtzche P. Sponsorship, bias, and methodology:
Cochrane reviews compared with industry-sponsored meta-analyses of
the same drugs. Proceedings of The 5th International Congress On Peer
Review and Biomedical Publication, Chicago, September 2005.
20 Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S. Why
olanzapine beats risperidone, risperidone beats quetiapine, and
quetiapine beats olanzapine: an exploratory analysis of head to head
comparison studies of second generation anti-psychotics. Am J Psychiat
2006;163:185–94
21 Fis˘ter K. At the frontier of biomedical publication: Chicago 2005. BMJ
2006;331:838–40
22 Chalmers I. In the dark. Drug companies should be forced to publish
all the results of clinical trials. How else can we know the truth about
their products. New Scientist 6 March 2004:19
23 Pearn J, Chalmers I. Publish and be applauded. New Scientist 6 January
1996:40.
24 Herxheimer A. Open access to industry’s clinically relevant data.
BMJ 2004;329:64–5
25 Zarin DA, Tse T, Ide NC. Trial registration at ClinicalTrials.gov
between May and October 2005. N Engl J Med 2006;353:2779–87
26 House of Commons Health Committee. The Influence of The
Pharmaceutical Industry, 4th Report of Session 2004–05. London:
Stationery Office, 2005
27 Angell M. The Truth About The Drug Companies: How They Deceive Us and
What To Do About It. New York: Random House, 2004
28 Vandenbroucke JP. Without new rules for industry-sponsored
research, science will cease to exist. Rapid response on bmj.com,
posted 14 December 2005 [http://bmj.bmjjournals.com/cgi/
eletters/331/7529/1350]
29 Kassirer J. On The Take: How Medicine’s Complicity With Big Business Can
Endanger Your Health. New York: Oxford University Press, 2004
30 Sim I, Chan A-W, Gu¨lmezog˘lu AM, Evans T, Pang T. Clinical trial
registration: transparency is the watchword. Lancet 2006;367:1631–3
341
J O U R N A L O F T H E R O Y A L S O C I E T Y O F M E D I C I N E V o l u m e 9 9 J u l y 2 0 0 6