corner
Healthy Skepticism
Join us to help reduce harm from misleading health information.
Increase font size   Decrease font size   Print-friendly view   Print
Register Log in

Healthy Skepticism Library item: 19469

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

 

Publication type: Journal Article

Lai NM, Teng CL, Lee ML
Interpreting systematic reviews: are we ready to make our own conclusions?: A cross-sectional study
BMC Medicine 2011 Mar 30; 9:(30):
http://www.biomedcentral.com/1741-7015/9/30/abstract


Abstract:

Background
Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors’ conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results.

Methods
We conducted two cross-sectional studies: one with hospital practitioners (n = 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (n = 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors’ conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors’ conclusions and labelled this as our best conclusion. We compared the participants’ choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial (“positive”), and two others did not (“negative”). We also asked the participants their prior beliefs about the intervention.

Results
Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (P < 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. “Positive” SRs were more likely than “negative” SRs to change the participants’ beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and “convert” those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8).

Conclusions
The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors’ conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different. Please see Commentary: http://www.biomedcentral.com/1741-7015/9/31/.

 

  Healthy Skepticism on RSS   Healthy Skepticism on Facebook   Healthy Skepticism on Twitter

Please
Click to Register

(read more)

then
Click to Log in
for free access to more features of this website.

Forgot your username or password?

You are invited to
apply for membership
of Healthy Skepticism,
if you support our aims.

Pay a subscription

Support our work with a donation

Buy Healthy Skepticism T Shirts


If there is something you don't like, please tell us. If you like our work, please tell others.

Email a Friend








Far too large a section of the treatment of disease is to-day controlled by the big manufacturing pharmacists, who have enslaved us in a plausible pseudo-science...
The blind faith which some men have in medicines illustrates too often the greatest of all human capacities - the capacity for self deception...
Some one will say, Is this all your science has to tell us? Is this the outcome of decades of good clinical work, of patient study of the disease, of anxious trial in such good faith of so many drugs? Give us back the childlike trust of the fathers in antimony and in the lancet rather than this cold nihilism. Not at all! Let us accept the truth, however unpleasant it may be, and with the death rate staring us in the face, let us not be deceived with vain fancies...
we need a stern, iconoclastic spirit which leads, not to nihilism, but to an active skepticism - not the passive skepticism, born of despair, but the active skepticism born of a knowledge that recognizes its limitations and knows full well that only in this attitude of mind can true progress be made.
- William Osler 1909