Healthy Skepticism Library item: 5264

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

 

Publication type: news

Strauss S.
Viewpoint : The virtue of being wrong
CBC News Online 2006 Jun 23
http://www.cbc.ca/news/viewpoint/vp_strauss/20060623.html

Notes:

Ralph Faggotter’s Comments:

Stephen Strauss details one of the areas in which science publishing can and does go wrong.

This malfunction leads to a distortion or bias in our perception of the effectiveness of medications because clinical trials which do not lead to a ‘positive outcome’ tend not to get published.

Full text:

STEPHEN STRAUSS:
The virtue of being wrong
CBC News Viewpoint | June 23, 2006 | More from Stephen Strauss

Stephen Strauss Stephen Strauss wrote articles, columns and editorials about science and technology for the Globe and Mail for more than 20 years. He has also authored three books, several book chapters, and for his efforts received numerous awards. Through all his time in journalism, he still remains smitten by the enduring wisdom of the motto of Austrian writer Karl Kraus. Say what is.

Here are some experiences you didn’t have today. You didn’t open your newspaper to discover a gargantuan front-page story recounting how it hadn’t snowed in Saskatoon yesterday.

You didn’t flick on the TV news and hear a commercial excitedly pronouncing that Canadian Tire was not reducing the prices of any of its barbeques.

You didn’t hear a radio report about the fact that for the 18,427th day in a row there was no civil war in Switzerland.

That is to say, almost nowhere in your media perusing life were you exposed to a null news story. “And this is bad?” I can hear you ask. “Are you suggesting I somehow need a regular dose of null news in my already too, too, too busy life?”

Well, maybe not you personally, I answer back, but that’s only because you aren’t a scientist. Science, at least theoretically, is rooted in the virtue of the null. Effectively, someone has a theory about how nature works and then, through experiment, tries to prove that theory is correct.

But more often than not the hypothesis pancakes out. Supposedly, what scientists then do is report their failures to a scientific journal which – also supposedly – then publishes a detailed description of what doesn’t work or isn’t so.

Null news is news because it allows other scientists to stop looking for truths in nature’s bogs and intellectual cul-de-sacs.

Only in reality the world’s scientific journals aren’t printing two negative to every five positive findings – in fact they are hardly ever printing null results at all. A 2004 study found that when 102 clinical trials were examined, researchers failed to report roughly 2/3 of results.

Yuck.

People who have studied the situation suggest null findings aren’t published because they are never submitted. And this is because:

1. The disappointed researchers want to leave their failures behind them as quickly as possible. 2. They don’t want to tell their competitors what they have been doing. 3. A failed result isn’t going to look good on your CV or get you tenure.

But it is also true that unless something seems absolutely, positively disproved, journals are loath to publish a null finding. Think of it as a journal ego thing. Instead of being the place where the cure for cancer was first described, the journal becomes the place where any number of failed cures were highlighted.

If non-publication of null findings were merely an ego thing, one could probably shrug and move along. But there are all kinds of evidence that in the absence of null publishing ignorance flourishes.

In retrospect, it now appears that the theory about stress causing ulcers kept being promulgated because null results weren’t published. The same with the widely and erroneously held belief that cellphone use leads to brain cancer.

So what to do? Well, in an age where the internet allows for cheap, widely available publishing, frustrated scientists have been setting up a few journals whose entire raison d‘être is the publishing of failure. Their number is not quite a flood yet, but they are a tiny wave.

The oldest is the Journal of Negative Results in Biomedicine. In it we recently learned that, try as they would, researchers at the Scripps Research Institute in California couldn’t find a link between gene defects, lung disease and exposure to the anthrax bacteria.

In the Journal of Negative Results – Ecology & Evolutionary Biology, British scientists turned their attention to the fact that eggs laid by a honey bee queen are allowed to hatch while eggs laid by workers are eaten by other workers. As far as they have been able to determine there is nothing in the shape or structure of the eggs that tells bees: Eat this one and leave this other one alone. When they looked for some kind of smell or taste signal, all the logical chemicals to look at came up blank.

Why have I been going on about this? Well, as you can see a few null result journals aren’t really addressing what looks like a big and likely growing problem. That’s where Nicole Stogaitis, a first year PhD student in chemistry at the University of Toronto, comes along. She is proposing a journal, tentatively titled “Roadblocks in Organic Chemistry” be set up in her field.

Her rationale is what we have described before: “Why do chemists waste their precious time and resources working hard towards a path that will inevitably fail? Because,” she says, “they have no way to know any better.”

Nicole came up with the proposal not knowing about many of the other null journals. But on reflection her idea also taps something quite patriotic in me.

Canada does not have – maybe with the exception of the Canadian Medical Association Journal – a crème de la crème scientific journal. That’s unlikely to change because we are a small country population wise. So what makes sense is that we specialize in something.

And what would be a better specialization niche for a place where the expression of national self-definition is as “cautious as a Canadian”, than for us to become the world’s centre of null hypothesis journals. It would be relatively cheap to do and think how natural and proud it would feel telling your children and your grandchildren: Because of Canada, this is what humanity now knows doesn’t work.

As I said, proud and natural and cautious as a Canadian.