Healthy Skepticism Library item: 17502

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: Electronic Source

Rind D
Evidence and Snake Oil
Evidence in Medicine 2010 Feb 14
http://www.evidenceinmedicine.org/2010/02/evidence-and-snake-oil.html

Notes:

Please visit the Evidence in Medicine to access links.

Full text:

Recently a friend wrote to me and asked what I thought of a presentation she had heard about a product containing beta-glucan:

Presentation part 1

Presentation part 2

Presentation part 3

I’m not suggesting anyone spend the time to watch all of this, but it is a 30 minute talk at a meeting (of salespeople, I think) of a company planning to market a version of an existing product with a preparation of beta-glucan added to it.

The speaker is excellent, and his audience is clearly impressed to have such a knowledgeable scientist speaking to them, and excited at the chance to market a new healthful product that the speaker promises will be one of the major revolutions in health in this half century or even century.

A claim like that is a bit astounding, and would obviously require high quality evidence to back it up. As Carl Sagan used to say, “extraordinary claims require extraordinary evidence”. That someone would make such claims at a sales meeting, such that the attendees were hearing about this revolution in health there first, rather than from the mainstream media reporting on a health breakthrough, might cause the incredulous to suspect they were about to learn about snake oil.

The credulous, on the other hand, might believe that meetings like this are exactly where health breakthroughs are presented because the mainstream media and mainstream doctors would never allow such information to be presented in a medium that they control.

The speaker talks of the astounding immune stimulating virtues of beta-glucan to repair the ravages that modern society has wreaked on our immune systems and points to a number of clinical trials. He goes on to describe the benefits shown in the clinical trials (mainly for upper respiratory tract infections — URIs) in glowing terms.

Since I’d been asked by a friend for my opinion, I decide to see what I could learn. I searched for clinical trials of the preparation for URIs and found a small randomized trial showing no apparent benefit of a beta-glucan product on URI in athletes.

Not surprisingly, this RCT had not been highlighted in the video. However, a number of other trials were mentioned and were linked on the product’s website. Two of the trials listed were apparently only presented in meetings, one was reported as published and I was unable to find the full text (perhaps others will have more luck), but one trial was published in a form I could access.

On the product website, the accessible study was described as showing:

While there were no significant differences in the incidence of symptomatic respiratory infections among the study groups, the duration and severity of symptoms were alleviated in subjects receiving Wellmune WGP.

In the study results, the Wellmune WGP group reported:

• No missed work or school due to colds, compared with 1.38 days of work/school missed for the placebo group. (p = 0.026)

• No incidence of fever, compared with 3.50 incidence in the placebo group. (p = 0.042)

• An increase in quality of life, including physical energy and emotional well-being, as measured by a clinically validated health survey questionnaire (SF-36v-2). (p = 0.042)

• No adverse events were detected and no safety concerns were present.

This actually tracks fairly closely with what the published study states in its abstract. However, if you actually read the paper it becomes quickly apparent that an enormous number of comparisons were made, nearly all of which showed no benefit to beta-glucan (the missed work or school due to colds was one isolated positive finding). The reported effect on fever does not match the data in the text of the article and, even there, no effect on fever was seen in the intention to treat analysis or in the per protocol analysis, though the latter got close to achieving that all important p<0.05 though it didn’t make it.
I communicated back to my friend my doubts that anything very interesting had been shown about the product and inquired about possible financial incentives influencing the interpretations of those who were reporting the results to her.

Why is any of this even worth thinking about? After all, the claims made in the video had raised my doubts as soon as I’d heard them, and there are many such claims for many products that are at least as dubious.

The reality, though, is that this beta-glucan product will be marketed and may gain traction. The friend who asked me about it actually currently sells an acai berry product in which she is a true believer. Acai berry products are being marketed all over the Web and doctors are now fielding questions from patients about their value. In my role as a medical editor, I was asked recently by a reader (a skeptical physician) to have our resource include an article discussing the evidence for and against clinical benefits from acai berry supplements.

If we wrote such an article about acai berry, what should it contain? There is no high quality evidence about acai berry and clinical outcomes. Should an article take an agnostic view about its benefits? Should it point out that lots of antioxidant supplements have failed to improve health or have even caused harm? Should it be written at all, or is there some threshold of believability or widespread use that must first be overcome.

The scientific medicine world generally claims it expects some reasonable evidence before administering a new preparation or trying a new procedure (though is not always as good in practice as in theory), and the pharmaceutical industry, for all its problems, can’t release a drug without impressive clinical trials.

But when random claims of snake oil salesmen selling some new product spring up, eventually, if the product really does become widespread in the marketplace, the expectation of many in the media and the public seems to be that the burden has shifted to scientific medicine to disprove the benefit or else not take a position about whether a product is good or bad.

At the extreme, you have the most ludicrous ideas of alternative medicine like homeopathy. But even more plausible-sounding ideas when untested do not deserve much credence. That someone who is selling a product can weave a good story about why it is beneficial should not require the resources of the rest of us to test those claims.

We’d all be better off starting out as skeptics, particularly when those promoting an untested product have a financial stake in its success. Snake oil salesmen have been a bane of appropriate medical care for centuries, and there’s no reason to think this will change. Those who care about science and evidence, though, must remember that we need not approach entirely untested claims from a position of neutrality. Most things you could imagine doing to improve health do not work, and many are harmful. Though absence of evidence is not evidence of absence, we need not spend time and thought disproving the claims of every snake oil salesman who comes along. It already takes enough resources testing just those claims that have some shred of plausibility as well as the products that have become so widespread that letting them stay untested endangers health.