Sunday, December 5, 2010

What we think we know?

We have acquired a lot of information on a subject, even some might say knowledge, and still know very little. That science is placed on a pedestal does not really help – perhaps it even hinders. After all, through science we expect to know things with confidence if not certainty. If it is proven through scientific experiment it must be true. Right? Wrong.


Scientific experiment even if it can be accurately repeated proves only the conditions tested by the experiment. We tend to be accepting of the results whereas we should, as with all things maintain an open mind and be sceptical.
This post is triggered by a recent discussion where someone said they had read at least 10 books on the subject and they all basically agreed and the experimental evidence proved X. My view, so what?
Consider a non-theological example and one which I have much sympathy for, namely that homeopathic medicine is a waste of money, being ineffective.
Experiment (1) to test the claims of homeopathy was conducted (I do not have the reference, but my point lies in the principle not the specific of the example). Students were infected with the common cold and while confined in the same accommodation were given (double blind) either a homeopathic remedy or a placebo. Conclusion, the homeopathic remedy was no more effective than the placebo. This experiment confirmed the results of similar studies and is seen as supporting the conclusion that homeopathic remedies don’t work.
Now I take exception to that conclusion and question the validity of the experiment. The experiment tested the clinical effectiveness of a substance – I would accept the results for a commercial cold medicine. Certainly it demonstrated that in a clinical trial the substance was not effective.
BUT that is not the claim made by homeopathy. Classical homeopathy claims to provide a holistic treatment of the person, not a symptom relief treatment as with a commercial cold medication. Therefore the experiment did nothing to test the basic claim of homeopathy.
An experiment that involved a full consultation with a homeopath then a triple blind supply of either placebo or the prescribed homeopathic remedy would certainly provide a better test for the claims. BUT is that still good enough?
Possible outcomes:
A: Either placebo or remedy have better outcome
B: Placebo and remedy have similar outcomes and similar treatment rates as for first experiment.
C. Placebo and remedy have similar outcomes but better treatment rates than for experiment (1). This would be an expected outcome. From what we know of the placebo effect it is reasonable to expect that the method of administration (thorough examination, diagnosis & prescription) would enhance the effectiveness of both placebo and remedy.
So what have we proved? I suggest that there would be evidence to suggest that the homeopathic method was more effective than the clinical method of experiment (1). But certainly one could argue that there is no control group that received a similar level of attention, from say a conventional doctor and then received placebo or remedy – let’s call that experiment (3). I would again predict outcome C.
So what has been proven. Perhaps we should compare the level of efficacy of the placebo in (2) and (3). BUT I still argue that we do not have a valid experimental methodology to test the claim of homeopathy that it is effective as a long term holistic treatment regime. Problem is I can’t think of a practical experiment to test this claim. Such an experiment (4)would have to:
i) Look at the long term effect on general health (how?)
ii) Ensure the equivalence of the health prospects of the participants at the start (how?)
iii) Maintain an equal health environment for participants throughout the experiment (how?)
iv) Provide a valid control groups – perhaps both placebo and conventional medication (in the light of i, ii & iii – not easy)
v) Direct the treatment towards condition(s) considered appropriate for homeopathic intervention – and how do all agree on that?
vi) Administer the homeopathic treatment in line with the homeopathic school, but the placebo in a neutral way (how? Given the responsiveness of the placebo effect to the method of administration).
Now I have found no experiment equivalent to (4) that I would consider a valid test of the claims of homeopathy. So where do we go now?
How about common sense? The homeopathic method has been around for a long time. Its practitioners (the scholarly ones anyway) will have developed their treatment and remedy regimes based on what was found to work (and not work). Their patients would have continued to consult them )or not) based on how effective they considered the treatments to be.
There is a history of scholarship (perhaps not of the type found in a prominent university science faculty) in homeopathy, its treatments are differentiated to align with patient needs and demonstrate a level of consistency. Patients have not unanimously decided it does not work.
Conclusive? Hardly. Of the treatment options available I still find more attractive options. But don’t expect me to accept - that because someone reads a book or two on the subject and that the simple experiments they contained “proved” it does not work – that indeed the case is closed ... and certainly, just because you read on the www or saw it on TV or heard the interview on the radio does not excuse you from not properly exercising your scepticism and thinking it through.
OK now who wants to talk about experiments that test the effectiveness of prayer?

Hamba kahle - peace

4 comments:

Pete Chown said...

Strictly speaking experiments only tell you something that happened on one occasion. Even if you duplicate the exact conditions, the results may have been a one-off: the universe might behave in different ways at different times.

Science is useful because the universe is generally quite predictable. It doesn't normally do different things at different times. When we've done a few experiments, we can often come up with a theory, which encapsulates an underlying physical law. If we do further experiments, they will often confirm the theory, because the universe is predictable. (If the experiments don't confirm the theory then the theory is invalid, of course.)

It's true that experiments can't always capture real world conditions. At that point, we can either give up the quest for knowledge, or we can test our theories with a variety of experiments that get as close to the real world as possible.

We've generally taken the latter approach, preferring some uncertainty to complete ignorance. In spite of this, the results have been impressive. To give one small example, you are typing the argument that science doesn't work on a computer that can only be built because of science. :)

I don't think anyone would claim that homeopathy doesn't work. They would claim that it is indistinguishable from placebo, which is a different thing. That is the reason why you can't just use 'common sense'. Common sense has the opposite problem to science. It considers the real world, with all its complexity and uncertainty. As a result, it often fails to distinguish different effects. In this particular case, it fails to distinguish placebo from pharmacological efficacy.

Finally, if you think homeopathy is real, why aren't you trying to find a way of measuring it? James Randi's $1M would be nice, and you could be pretty sure that the Nobel prize would follow shortly after. Science honours people who disprove its dogma.

akakiwibear said...

Hi Pete. I agree with you. This was in part a rant at those draw improper conclusions from experiments.

I would love his $1m ... but actually I am not a homeopathy fan.

Would you like to take up the challenge and devise an experiment to test if prayer works?

The results from the various experiments (all of which are in my mind flawed) are inconclusive anyway - including showing prayer to have a negative effect!

Hamba kahle -peace
P.S. your comment ended up in the SPAM folder? Usually a sign that your email address has been "borrowed" by a spamer :(

Pete Chown said...

Faith healing apparently qualifies for the Randi prize, so you could win it that way too.

It might be possible to design an experiment which shows that prayer works (if that is the case). Showing that prayer never works is probably impossible, I think. Perhaps God simply doesn't like theological experiments, so prayers offered in that context always fail.

I suppose the point is that a personal god can behave in arbitrarily complex ways. If we were testing a new drug, for example, we would assume that the drug's pharmacological effects are not influenced by the fact of the patient being part of an experiment. That seems like a reasonable assumption in the case of a drug, but not in the case of God. We cannot, of course, verify it in either case.

Not being a Christian, I don't feel I'm the best person to design an experiment which could demonstrate the benefits of prayer. I see that various experiments have failed to demonstrate benefit, but I don't feel qualified to say what the reasons might be, apart from the obvious one that there is no God.

Incidentally, I find this experiment particularly discouraging:

http://www.religioustolerance.org/god_pra2.htm

'God' seemed to have no settled view. Each person who asked His guidance was told what he or she wanted to hear. We've already talked about various reasons why these experiments may not be valid, but these results do fit perfectly with the atheist belief that prayer is a way of talking to yourself.

akakiwibear said...

... ended up in spam again ... ? I agree with talking to yourself ... the science is sooooooo bad it defies belief!

Most experiments related to healing or clincila performance of similar patient groups. Some experiments actually showed a significant negative impact on treatment effectiveness from prayer - so prayer must work ;)

I think the real problems with the medically based experiments are:

1) What is actually an answer to prayer - most Christians would agree that spiritual healing would be expected rather than a physical healing miracle.

2) A control group is almost impossible to find as there are religious orders that pray for everyone (worldwide, all religions) who is sick ... and just about every other adverse circumstance too.

3) Current Christian theology prefers the view that we act as God's agents in answering most prayer - God kicks our (our general or a very specific our, say an aid worker or medical professional) butt to do something for someone. This leaves only the actual miracles as acts of God in responce to prayer ... and if they were common and/or predictable they would not be called miracles.

Hamba kahle -peace