Tuesday, July 24, 2007

But can it make your hair grow in three weeks?

Wow. Welcome to the crazy. Today's article is An investigation into the effect of Cupping Therapy as a treatment for Anterior Knee Pain and its potential role in Health Promotion by Ahmed Younis et al. It was published in the estimable Internet Journal of Alternative Medicine, so you know it must be true. At any rate, you know it must be freely accessible online, which is terrific. Such a service to humanity.

If you don't know what cupping is, that's probably because you were born sometime after the dawn of the 20th century. Cupping is... well, let's just ask the authors:
Cupping is an ancient method of treatment that has been used in the treatment and cure of a broad range of conditions; blood diseases such as haemophilia and hypertension, rheumatic conditions ranging from arthritis, sciatica, back pain, migraine, anxiety and general physical and mental well-being. The aim of Cupping is to extract blood that is believed to be harmful from the body which in turn rids the body of potential harm from symptoms leading to a reduction in well-being.

Now, I know what you're thinking*: they're performing bloodletting? Is skepstat frakking kidding me? I am not frakking kidding you, but the authors are when they claim in their conclusion that:
The efficacy of the treatment of Cupping for Anterior Knee Pain, Range of Movement and well being has been researched and results reveal statistically significant differences in support of Cupping Therapy.

The gist: The authors sucked some blood from 15 participants complaining of knee pain and measured their range of motion and obtained self-reports of pain and well-being before the bloodletting, excuse me, cupping therapy, and three weeks after the bloodletting. I'm sorry, cupping therapy.

The results: Improvement in outcomes across the board! Low p-values! Huzzah!

The problems: As you should expect, there are any number of ginormous problems with the study. We'll cover the two worst:

  1. No control group. Let's engage in a thought experiment. Suppose you take 15 random people complaining of acute knee pain (from any source, mind you), and ask them how their knee feels. They'll probably say it hurts. In fact, they already did. Now wave a kosher dill pickle at them and ask them again 3 weeks later. Chances are, they'll say it doesn't hurt quite so much. This could be seen as an example of what statisticians call "regression to the mean," but it's probably more accurately described by what doctors like to call "getting better."

    This is why you always need a control group in treatment evaluation. There's absolutely no way to tell how much of the improvement is due to the experimental therapy if there's nothing to compare it to. In some circumstances you might want a control group that receives absolutely no intervention, in some cases a control group that receives placebo (not the same thing as no intervention, by the way), in some cases a control group that receives a different active intervention. But you always need a control group.

    In other news, non-controlled studies have now shown that cupping therapy makes the sun set at night and then rise again in the morning.

  2. Missing data. This is a problem that haunts legitimate research as well, but it's particularly bad in this case. The authors recruited 26 potential victims, but only 15 completed the trial. What happened to the other 40% of the sample? Well, 4 of them never showed up (maybe their pain got better?) and 7 didn't show up for follow-up (maybe their pain didn't get better? maybe they became afraid, so afraid of the fake doctors with the real razor blades?). Real scientists would have tried to include the 7 follow-up no-shows in their analyses, often by assuming that the missing subjects had no improvement at all (a so-called last-observation-carried-forward or LOCF analysis, although most statisticians will tell you that LOCF analyses are terrible). Luckily for the bloodletters, this isn't a real journal.

Favorite line: Cupping Therapy has no major side effects aside from minimal discomfort due to the method of application of skin cuts to the patient. In cases where the patient's pain threshold is low, a local anaesthetic can be administered.

Least favorite line: Ethical approval was sought from Kings College Research Committee.

You Brits really need to get your ethical act together.


* Rhetorical device. I am not** actually using psi powers to discover what you are thinking.


** Not currently.

1 comment:

furryyuri said...

Who do I have to pay to get another post?