But I'm back, with a treat. Today we'll be enjoying a woo-woo challenge to clinical trials methodology, in the form of Resonance, Placebo Effects, and Type II Errors: Some Implications from Healing Research for Experimental Methods, published in The Journal of Complementary and Alternative Medicine by William F. Bengston ( an energy healing sociologist) and Margaret Moga (an anatomist and moxibustion enthusiast).
Randomized controlled trials
The randomized controlled trial (RCT) is generally considered to be the gold standard experimental design for medical research. RCTs are widely used and there's a huge literature on RCT methodology (including a journal entirely devoted to the subject), but the basic idea is quite simple.
Let's take a relevant example. Suppose we want to examine the efficacy of, say, laying-on-of-hands "energy healing" for shrinking tumors in mice. Well, efficacy is always efficacy relative to something, so we need to choose an appropriate control treatment. In this example, let's say we're interested in comparing "energy healing" with no treatment at all.
Well, we need to start with a sample of mice with tumors. In animal research, when you want to study animals with a certain disease, you usually start with healthy animals and then give them the disease you want to study. (Generally, this practice would be looked at somewhat unfavorably in human subjects research.) So, we'll take a bunch of mice and inject them all with tumor cells. Stop looking at me like that, animal rights activists. Anyway, maybe one of the experimental mice would have gone on to become the mouse equivalent of Hitler. You never know, do you?
Anyway, the next step is key: we randomly assign the cancer mice to receive either the "energy healing" or no treatment at all. The principle of randomization is deceptively simple, but it has profound implications. The most immediate implication is that, because all personal characteristics of the subjects are randomly distributed between the groups, any observed differences in outcome are likely to be due to the treatment assignment rather than to irrelevant factors like age, sex, socioeconomic status, or whatever the mousy analogues of these might be. It's no exaggeration to say that the introduction of randomization by (principally) R. A. Fisher, was one of the seminal moments in the history of science.
Then we give our treatment group the treatment of interest (i.e., laying-on-of-hands) and our control group the control treatment (i.e., no treatment). After an appropriate length of time, we measure the tumors on all our mice, calculate the remission rate for the treatment group and the control group, and use standard statistical methods (Fisher's exact test, say, although more on this subjet another time) to decide whether the observed difference in remission rates is larger than would be expected due to simple chance differences between the groups.
The inconvenience of negative results
Now suppose you conduct the experiment described above not once, not twice, but four separate times, each time with the same result: many of your "energy healing" treated mice undergo tumor remission, but so do many of your control mice. In fact, in each experiment, the difference between treatment and control groups is smaller than would be expected due to chance ("not statistically significant").
Now, a few possible explanations might spring to mind for these negative results:
- The tumor induction was inadequate in both control and treatment mice to result in a sustained cancer. The majority of mice remitted because their disease was self-limiting. The "energy healing" treatment doesn't work better than no treatment.
- The tumor induction was adequate but some physiological process or trait separate from the treatment but common to all mice resulted in tumor remission in the majority of mice. The "energy healing" treatment doesn't work better than no treatment.
- #1, except the "energy healing" treatment does work better than no treatment, but the effect is small and the sample size was inadequate to determine statistical significance.
- #2, except the "energy healing" treatment does work better than no treatment, but the effect is small and the sample size was inadequate to determine statistical significance.
- The study protocol was broken in some way due to carelessness or fraud.
- THE VERY ACT OF RANDOMIZATION CREATES A MYSTICAL RESONANT BOND BETWEEN THE TREATMENT AND CONTROL MICE WHICH MEANS THAT ANY ENERGY HEALING APPLIED TO THE TREATMENT MICE IS TRANSFERRED TO THE CONTROL MICE. THE TREATMENT WORKS SO WELL THAT IT WORKS EVEN ON MICE THAT DIDN'T ACTUALLY RECEIVE IT!!@$!@@$!!
That's right! Number 6 is obviously gold. The really great thing about the hypothesis underlying number 6 is that, the more negative your results are, the more amazingly effective your treatment must be! The only way things could look better for the treatment is if more control mice remitted. And not only does the hypothesis explain this experiment, it single-handedly accounts for all placebo effects in all clinical trials, ever. This is paradigm-shattering stuff, truly.
But let's not jump to conclusions. We're scientists here. We like the cut of hypothesis 6's jib, sure, but we're not going to just jump in and publish an article proposing an occult relationship between control and treatment groups in RCTs without first making damn sure we can back it up with science. What we need is an experiment....
How do you test an insane hypothesis?
Why, with an insane experiment of course. SILLY! AHAHAHAHAHAAHAHA.
Ok, so here's what we'll do (and, in case you haven't caught up with my rhetorical stylings so far in this post, "what we'll do" translates to "what Bengston and Moga did"). We'll take 30 mice, inject them with tumor cells, and randomly assign half to "energy healing" and half to no treatment. So far so good - it sounds like a replication of our previous experiments.
But wait, you're saying, what about the resonant bond? If our revolutionary hypothesis is correct, the two groups will be mystically connected, all the tumors will remit, and we'll be back to square one. What we need is a TRUE CONTROL GROUP that won't be bonded with the treatment group, resonantly or otherwise. Then we can see if our two original groups are more similar to each other in course of disease than they are to the third control group.
But how to make that third, TRUE control group? Well, why not take another 25 mice and not inject them with cancer cells at all in the first place? Just 25 mice. No tumors. Then we'll follow all three groups, see if remission rates and biological markers are similar in the treatment and bonded control groups and different in the group that, hey, we never gave cancer in the first place. Why this group should be immune to the mystical resonant bond is anyone's guess, of course, but it's worth a shot, right?
Well, ladies and gentlemen, 55 innocent (or, who knows, maybe not so innocent) mice and several inappropriate statistical analyses later (the rest of this is so much fun, I can't even be bothered to critique their statistics), our long hours of work have paid off. Remission rate in the "energy healing" treatment group? 100%. In the bonded control group? 100%, that's all. Just a paltry 100%. And what about the third control group? The group that NEVER HAD CANCER IN THE FIRST PLACE? 0%. Zero. The big zilch. Nada. Not a single one of the mice that NEVER HAD CANCER IN THE FIRST PLACE remitted from their cancer.
Just in case that's not enough to convince you, Bengston and Moga also measured hemoglobin levels and weighed the spleens of a subset of mice from each group at each follow-up. Guess what? The group of tumor-injected mice that had "no" treatment and the group of tumor-injected mice that had "energy healing" treatment? Pretty much the same. And the group that never had tumor cells injected? Different!
An analogy
The setup: In case all of this talk of mice and tumors has gotten a bit esoteric, here's an analogy for what we've just learned. Let's take three cars: car A, car B and car C. We fill the fuel tank of car A with gasoline, plus a fuel additive. We fill the fuel tank of car B with gasoline only. We leave the fuel tank of car C empty.
The experiment: We line the three cars up and race them, to see which can go farthest in 10 minutes. Cars A and B go 20 miles each. Car C doesn't go anywhere.
The conclusion: The fuel additive is effective, and a resonant bond caused Car B to go as far as Car A. The existence of this resonant bond is proved by the fact that Car C didn't go anywhere.
And that just about settles that. Take a bow, Drs. Bengston and Moga. You've really done something here. Really.
Now those of you still reading shall be rewarded with a collection of choice excerpts.
Questioning the logic of experimental design is the last thing we want to do:
This paper does not question the logic of experimental
design. Rather, it suggests that, under some circumstances,
for example, illustrated by placebo effects, the presupposi-
tion of experimental and control group independence is
questionable. It suggests that this violation can occur via the
creation of a “resonant bond” between groups. Resonance,
in turn, can result in a macroscopic entanglement of exper-
imental subjects, so that a stimulus given to one group also
stimulates the other group.
Weeks of practice to master:
As previously reported, the healing-with-intent experi-
mental protocol required that the volunteer healers practice
mental and “directed energy” techniques taught to us by an
experienced healer formerly based in Great Neck, New York.
These techniques did not involve focused visualization, med-
itation, life changes, or belief of any sort. Although they are
straightforward, the mental techniques required weeks of
practice to master and involved a series of routine mental tasks
that were to be practiced simultaneously while placing hands
around the standard plastic mice cages for 1 hour per day.
Our curiosity got the better of us:
Our intent was to keep the control mice
separate for the duration of the experiment and to keep them
particularly hidden from anyone who knew the healing tech-
niques. Our curiosity got the better of us, however, and, within
several weeks of the first experiment, we violated protocol
and visited the control mice. In hindsight, this may have
proved fortuitous, because it inadvertently opened the door
to unexpected phenomena.
Naturally, it's all down to quantum entanglement:
Almost all of the seeming paradoxes of these remissions
disappear if we allow for the possibility of “resonant bond
formation” and “resonant bond dissolution,” which may
serve to entangle or de-entangle subjects. Certainly the no-
tion of “entanglement,” although still quite mysterious, is
widely accepted and hailed for its predictive power on a
quantum level in conventional physics.
You want an explanation of resonant bonds? Well, how about two explanations, smarty-pants?
Consider two possible hy-
potheses: (1) shared experiences among experimental sub-
jects can “bond” them together resonantly; and (2)
consciousness itself, including that of the experimenter, can
delimit the boundaries of experimental subjects, effectively
defining those who are “in” and those who are “out.” Those
who are “in” form something akin to a larger “collective,”
analogous to those formed by colonies of insects, flocks of
birds, and schools of fish.
All your failed studies are belong to us:
Researchers are encouraged to
reexamine their old data within the framework of resonance
to determine whether these phenomena are as extensive as
they now appear to be (e.g., placebos). This reexamination
needs to broaden the question from the difference between
experimental and control subjects to inquire more generally
about the difference between experimental subjects and
“what ought to have happened.”
