Check the cherries you pick – they might be rotten

My attention was belatedly drawn to the BBC News item in which the President of the Faculty of Homeopathy, Dr Sara Eames, claimed that there were over 100 clinical trials that provide good evidence that it works. Sadly her opponent in the debate was well-meaning but very poorly informed. The reality is that over 200 clinical trials of homeopathy are in the literature, so was Eames picking the 100 best ones? Well she would have to pick far fewer than that.

But my mind was thrown back a year, when I debated with Eames on a BBC radio programme. The recording has long since disappeared from BBC iPlayer, so you’ll excuse me if I rely on memory. At the time I pressed Eames to cite just one disease in which there was good evidence for homeopathy. After a lot of prevarication, eventually she picked menopausal hot flushes.

After the programme I emailed her and asked for a reference to an RCT in hot flushes. I had to chase her, but she replied with the following:

I too was pleased to have the opportunity to discuss homeopathy. The study which I had in mind when talking during the programme is

Beer, A.M., Sturm, R., Kupper, F. Der Einsatz eines homoopathischen Syndrom im Vergleich zur Hormonsubstitution.

Erfahrungsheilkunde 1995; 44: 336-340.

ABSTRACT: In a randomized multi-center-study 130 patients suffering from menopausal syndrome were treated. 64 patients received a hormone-substitution and 66 patients were treated with a complex homeopathic remedy. The results indicated that in comparison to the hormone-therapy the FSH-,E2- levels, the cytologic results and the thickness of the endometrium have shown no significant changes. The subjective complaints however became significantly better under homeopathic treatment which is comparable to hormone-therapy. Therefore homeopathic therapy is supposed to be an alternative for patients objecting to hormone-therapy or for patients showing contrindications against hormones, but who want to alleviate their menopausal symptoms,

With Best Wishes,

Sara

So of course I looked up the paper, and found that it was in German. A certain German-speaking professor of my acquaintance (and of the right specialism) read it for me. Here is what he said:

1. very badly reported study
2. should have been a non-superiority trial but I am not sure that it was designed such
3. no info on patient recruitment: if patients were invited for a trial of homeopathy, those receiving hormones would be disappointed and hence have unusually poor outcomes (the trial was not blinded, and there was no placebo group)
4. most results were within groups
5. very few differences between groups
6. multiple testing for significance without correction
7. Kupfermann Index -0.45 (hormone replacement) and -0.32 (homeopathy)
8. the homeopathic remedy contained several ingredients in material dilutions (e.g. Pulsatilla D2!)

Conclusion: a very poor study providing no sound evidence.

The interesting thing is that this example isn’t really cherry-picking, because the one she picked was rubbish. But a radio listener could not know whether the `evidence’ was valid or not, and would assume it was. So is Dr Sara Eames simply deluded or being less than scrupulously honest? Bear in mind that she is a qualified medical doctor in general practice. She is obliged under her registration with the General Medical Council to follow evidence based practice. Yet she has put her name to the advice sheet on influenza issued by the British Homeopathic Association. Needless to say there is no evidence at all for the claims in this sheet. Yet the terms of GMC registration require that all doctors must “provide effective treatments based on the best available evidence”.  This really means that all medical homeopaths are in breach of their terms of registration and should be reported for disciplinary action.

It’s also interesting that, in a written reply to me from Dr Graham Archard, research adviser for the Royal College of General Practitioners, he said:

When it comes to homeopathy the College is not convinced that there is sufficiently robust evidence to support its general use. Indeed the National Electronic Library for CAM (NELCAM) seems to suggest there is little evidence for its efficacy. Thus currently the College would not generally support its use. If the evidence base changes we would, of course, be happy to reconsider.

That was in 2006. Has the evidence based changed? If anything it has further confirmed homeopathy’s inefficacy. I have not been able to find out whether Dr Eames is a College member, but in any case I have never heard of the College having disciplined any member or fellow who practises homeopathy. This leads me on to the whole area of regulation which I’ll address another time. Meanwhile, what would happen if members of the public complained to the GMC about all doctors who are homeopaths? That might be interesting.

When it comes to homeopathy