The gist: The authors randomly assigned 30 young men to two groups. Group 1 received electroacupuncture (that is, acupuncture with a weak electrical current delivered in continuous pulses through the needle) in their right leg three times a week for four weeks. Group 2 received no intervention. Dorsiflexion (foot elevation) strength was measured for each subject, in each leg, before and after the four week period, by means of a homemade device.
The results: Well, how about a mean 21.3% increase in strength in the right legs and a 15.2% increase in strength in the left legs of Group 1, compared to a measly 3.0% right leg and 4.8% left leg increase in Group 2? How's that for impressive, Mr. / Ms. Skeptic? Someone alert Barry Bonds!
The problems: Refreshingly, the statistical analysis was fairly reasonable and appropriate. The authors used a repeated-measures analysis of variance, with timepoint (pre vs. post), leg (right vs. left) and group as factors. There isn't enough detail to fully evaluate their statistics, but the basic idea is correct.
No, the main problem here was blinding. As in, there was no blinding. What does this mean? It means that, at all stages of the experiment, all the participants and all the investigators knew who was getting electroacupuncture and who wasn't. Why is this a problem? Because biases (both conscious and subconscious) on the part of the participants and the investigators can dramatically skew results.
Participants in Group 1 knew they were getting a treatment that was supposed to increase their strength, and they knew that the investigators wanted them to have increased strength at the end of the trial. This is more than enough motivation for most of these participants to try really hard at the post-intervention strength test. Naturally, individual strength performances can be strongly influenced by motivation and willpower. In the extreme, it could even have been enough motivation for some participants to, gasp, exercise in between sessions. Conversely, Group 2 knew that they weren't supposed to get stronger and that the investigators didn't want them to get stronger.
And as for the investigators: they could have been sending subtle (or not so subtle) signals to the participants to try harder (Group 1) or not so hard (Group 2) during the post-intervention strength test. This problem could have been easily mitigated by using blinded, independent assessors for the second strength test. Sadly, this was not the case.
Oh, and one last smallish problem with the study: utter biological implausibility. I mean, come on, you stick needles in the right leg and the left leg gets stronger? Scientist, please.
Favorite line: We did not collect data with respect to the balance of qi, whereas the subjects in this study were apparently healthy.
No comments:
Post a Comment