Healthy Skepticism Library item: 11656

Warning: This library includes all items relevant to health product marketing that we are aware of regardless of quality. Often we do not agree with all or part of the contents.

Publication type: news

Ten Things Wrong With Medical Journals
The Last Psychiatrist (blog) 2007 Sep 30
http://thelastpsychiatrist.com/2007/09/ten_things_wrong_with_medical.html

Full text:

I know, right? Only ten?

References

This is how references are done now:

…the Framingham Heart Study, HDL cholesterol
level was more potent as a risk factor for coronary
heart disease than was the level of low-density
lipoprotein (LDL) cholesterol.4 An analysis of data
from four large studies concluded that each in-
crease of 1 mg per deciliter (0.03 mmol per liter)
in HDL cholesterol is associated with an decrease
of 2 to 3% in the risk of future coronary heart
disease.6 …

———————

…
4. Gordon T, Castelli WT, Hjortland MC,
Kannel WB, Dawber TR. High density lipo-
protein as a protective factor against coro-
nary heart disease: the Framingham Study.
Am Med J. 1977;62:707-14.

6. Gordon DJ, Probsfieldt JL, Garrison
RJ et al. High-density lipoprotein chole-
sterol and cardiovascular disease: four pro-
spective American studies. Circulation
1989;79:8-15…

This is inane.

I could make a career out of exposing references that have nothing to do with, or directly contradict, the referenced statement. (Here’s one. http://thelastpsychiatrist.com/2006/03/is_schizophrenia_really_bipola.html) This system of referencing makes it very hard to do this— and I wonder if that’s not the point.

It is, by contrast, very easy to link the exact article referenced— even the exact page in the article. There is no reason not to do this. Even in this ridiculous blog, I link back to at least the abstract of each article I reference.

One might respond that, online, when you click on the superscript, it takes you to the references, which has a further link to the actual article. Ok— why the extra step?

Note that the way references are done here is antithetical to science. Look at the reference pictured above. What do you see? What is important? What you see are the authors and the journal, not the scientific content. That’s what’s implied to be important. We’re supposed to accept the science of a statement by the force of the author and journal? But that speaks to my later point about bias.

Subscription

Sorry, why should government or Pharma sponsored research require me to pay someone else (the Journal) for access to that information? So, if I’m a welder in Kentucky, I can’t know what’s really up with Depakote for bipolar? I have to read some nut blogger?

It is a simple process for a Journal to host online all articles as free text. Or, better, scientists can publish their work on their own site (hosted by a university, etc, if necessary.)

Neither are Journals necessary as repositories of vetted information. There are numerous ways these scattered articles can be collated, packaged and even summarized for easy use. Slashdot, Digg, and others are very effective in this regard; and something similar can be done with science. I know, Digg can be gamed. What, Journal of Clinical Psychiatry can’t?

Where’s The Raw Data?

Rephrasing from above: why am I not allowed to see the raw data from a government sponsored study? (And from Pharma— if they agree to do a study, then they must agree to make all data public.)

You may have heard that there are rumblings about making this data accessible— but not to everyone; only to those with appropriate access (academics, etc.) Again— why?

Slow

This is a common lament, but it misses the point: it is artificially slow. As in on purpose.

Articles that are submitted for peer review should simply be published in a “pending peer review” section. Other sciences already do this. To the criticism that doctors may act on unreviewed science, it should be noted that citing “personal communication” (e.g. an email) as a reference is perfectly acceptable. Is that safer? Oh, and about that peer review:

Peer Review

Even with a Cray XT4 supercomputer, MATLAB, the host of Fresh Air and a team of Vogons, it would be nearly impossible to create a worse system than peer review. A skull resection would be a cleaner way of evaluating information. Peer review does not have the potential for bias; it is specifically designed to retain bias, and to maintain the primacy of subjective opinion over objective findings. The only people who support peer review are other peer reviewers, and angler fish. If necessary, money should be diverted from pediatric AIDS research and however many koalas sacrificed in order to put a stop to this oligochracy. It’s that important.

In medicine, “peer review” is the editor of a journal and three other doctors- that the author suggests as reviewers. While ostensibly the author’s identity is unknown to the reviewers, in practice it is simple to determine authorship (type of research; meetings; and even looking at the Microsoft Word manuscript’s Properties.) Oh, and the editor knows who you are.

Most people think peer review is some infallible system for evaluating knowledge. It’s not. Here’s what peer review does not do: it does not try to verify the accuracy of the content. They do not have access to the raw data. They don’t re-run the statistical calculations to see if they’re correct. They don’t look up references to see if they are appropriate/accurate.

So what do they do? They look for study “importance” and “relevance.” You know what that means? Nothing. It means four people think the article is important. Imagine your four doctor friends were exclusively responsible for deciding what you read in journals. Better example: imagine the four members of the current Administration “peer reviewed” news stories for the NY Times.

On the force of the recommendation of these three reviewers, and the editor— who, by the way, decides whether to even send it to reviewers at all, or simply reject it— the article gets published or not. And there is no right to an appeal.

Imagine a movie gets previewed by four people, who decide if the movie is important or not, and whether it will play in theatres. You know what you get? Notes On A Scandal, that’s what. And riots.

The peer review system also promotoes the perpetuation of biases. Doctors are subtly pressed into writing articles about certain topics— consider the Depakote madness of 2000-2004; the noradrenergic hypothesis of depression in the 70s (where’d that go?); and how every issue of BMJ has an article on war. (Except July: that was the month they wrote about whether to boycott Israeli academic institutions. Ok.) Academic careers are made, in part, by the number and quality (i.e. journal) of publications, which will be influenced by what they think certain journals would publish.

Lack of Debate

There is no way to have a meaningful debate about an article within the Journal system. As an example: if I find a logical error in an article (e.g. mistaking correlation for causation) I can only point this out by writing a “Letter To The Editor.” Here’s the problem: “Letters” are peer reviewed, too.

The original article may be disseminated and read widely; but even if it is published, my Letter will have little attention. An example from my own practice: I’ve been as vocal as I can about how there is only one double blind, placebo controlled study about Depakote in maintenance bipolar— and it did not beat placebo. This fact is important precisely because it is popularly (by psychiatrists) believed that Depakote has volumes of such data supporting its efficacy. I don’t have to be a major academic player to look this fact up; but I do have to be in order for anyone to publish my discovery.

Contrast this with Metafilter or Slashdot. Not only is there meaningful debate, but one gets to know the participants (and can search on them.) There is a system for favoriting comments, etc. And the author can fix the article quickly to take into account any new information/criticisms.

Disclosure of Conflict of Interest

Almost completely invalid for its intended purpose.

So if a doctor does a dinner program for a drug rep, he has to disclose the relationship. But if a doctor is dating a drug rep— that relationship he doesn’t need to disclose.

Even more strange is that these are commercial conflicts of interest, only. If you are a communist, or Priest of Scientology, or a serial pedophile; these are not disclosures. Neither is being funded by the NIH (any surprise that NIH studies always find that the generic is the best?) Or being married to the Chair at Harvard. If your academic career is so far built on studying antiepileptics in bipolar, and you write an article saying lithium should never be used, only antiepileptics— isn’t that a conflict?

Aren’t personal beliefs a bias?

I’m not saying we shouldn’t disclose drug company ties; I’m saying to single those out as somehow more damaging, more biasing than any others is preposterous. It’s not a false sense of security; it’s a deliberate misdirection from all the other things that actually bias science. And it sidesteps the entire point of scientific articles— if they are truly scientific, if the articles were truly “peer reviewed”— it shouldn’t matter what your biases are. I could own Pfizer. The article on Zoloft should stand on its own.

It’s also worth noting that the peer reviewers are not asked to disclose any of their commercial interests.

Boring

Jesus. No exposition needed. I’m not saying you need references to Cheap Trick or a gratuitous nude woman with a bottle, but come on.

Abstracts As Promos

See this blog post, where it starts out, “I know, right: only ten?” and then you have to click to get the full article? So my promo has really no useful information in it. You know why? Because I am a blogger, that’s why.

Contrast that with the abstract from an important study on Lamictal for maintenance treatment in bipolar (emphasis mine):

Conclusions Both lamotrigine and lithium were superior to placebo for the prevention of relapse or recurrence of mood episodes in patients with bipolar I disorder who had recently experienced a manic or hypomanic episode. The results indicate that lamotrigine is an effective, well-tolerated maintenance treatment for bipolar disorder, particularly for prophylaxis of depression.

The Conclusions seem to say Lamictal is good for preventing mood episodes, mania and depression— is there any other way to interpret it? In fact, this study shows it is only good for preventing depression, not mania at all. Why is it written this way? Because the authors want to advance the idea that Lamictal is a “mood stabilizer” and not what it actually is: an antidepressant.

You have to understand that most doctors do not read the study, they don’t even read the abstract— they skim the abstract. “Look- just tell me what happened.” For this reason, the abstract has to be an accurate summary of the article, not a promo for an idea. But that’s why it is written this way; it’s not about the findings, it’s about the authors’ agenda.

What’s stupid about this is that negative findings are as important to a clinician as positive findings. They are less important, of course, to academics whose careers depend on positive findings, and the drug companies who sponsor them.

Healthy Skepticism Library item: 11656

Announcements