Healthy Skepticism Library item: 15775

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Liar! Liar!
Economist.com 2009 Jun 4
http://www.economist.com/science/displayStory.cfm?story_id=13776974

Abstract:

Scientists are not quite as honest as might be hoped

Full text:

THAT people, from politicians to priests, cheat and lie is taken for granted by many. But scientists, surely, are above that sort of thing? In the past decade the cases of Hwang Woo-Suk, who falsely reported making human embryonic stem cells by cloning, and Jan Schön, a physicist who claimed astonishing (and fabricated) results in the fields of semiconductors and superconductors, have shown that they certainly are not. However, on these occasions the claims made were so spectacular that they were bound to attract close scrutiny, and thus be exposed eventually. In the cases of Dr Hwang and ex-Dr Schön, the real question for science was not whether it harbours a few megalomaniac fantasists, but why the frauds were not exposed earlier when the papers that made the claims were being reviewed by peers.

Lower-level fraud, however, is much harder to detect: the data point invented or erased to make a graph look better, or to make a result that was not quite statistically significant into that scientific desideratum, the “minimum publishable unit”; the results “mined” retrospectively for interesting correlations, rather than used to test pre-existing hypotheses; the photograph that has been “enhanced” to bring out what the researcher regards as the salient features. How often this sort of thing happens is hard to say. But Daniele Fanelli of the University of Edinburgh thought he would try to find out. His results, published in the Public Library of Science, suggest it is commoner than scientists would like the rest of the world to believe.

Dr Fanelli’s own laboratory was the internet. He hunted down past surveys of scientific honesty and subjected them to what is known as a meta-analysis. This is a technique that allows the results of entire studies, which may not have used the same methods, to be pooled in a statistically meaningful way. Dr Fanelli found 18 surveys that met the criteria for his meta-analysis, and a few others that he also included in a general review.

Admissions of outright fraud (ie, having fabricated, falsified or modified data to improve the outcome at least once during a scientific career) were low. According to the meta-analysis, 2% of researchers questioned were willing to confess to this. But lower-level fraud was rife. About 10% confessed to questionable practices, such as “dropping data points based on a gut feeling” or “failing to present data that contradict one’s previous research”-though this figure was just a straight average of the underlying studies, since the relevant parts of the underlying studies were too disparate to run through the meta-analysis.

Moreover, when it came to airing suspicions about colleagues, the numbers went up. The meta-analysis suggested that 14% of researchers in the underlying studies had seen their colleagues fabricate, falsify, alter or modify data. If the question was posed in more general terms, such as running experiments with deficient methods, failing to report deficiencies or misrepresenting data, the straight average suggested that 46% of researchers had seen others get up to such shenanigans. In only half of the cases, though, had the respondent to a survey tried to do anything about the misconduct he said he had witnessed.

How much this actually matters is moot. Fabricating data is a heinous scientific sin. It steers people down paths that do not lead anywhere and discourages them from following those that do. But cleaning data up has a long tradition. Robert Millikan, the physicist who first measured the charge on the electron, discarded results that did not match his expectations, yet he won a Nobel prize-because he was right. The results of Gregor Mendel, the father of modern genetics, are also suspiciously over-accurate by the tenets of modern statistics. When such practices shade into dishonesty is itself a shady area. Just as everyone thinks himself a better-than-average driver, these results (assuming that they are honest) suggest people are more willing to see sin in others than in themselves. And that, at least, proves something that is sometimes forgotten. Scientists are as human as everyone else.