Evidence: It’s whatever you want it to be

I have just sat through the an oral session for the evidence check on homeopathy, called by the House of Commons Science and Technology Committee. Well I wasn’t there in person, I watched it on the excellent video page. You can still see it if you are quick. Before I go on, aren’t we all glad they have reverted to a sensible name for this committee? Anyway, it was almost two hours very well spent. The transcript will be what endures for the public record, but that doesn’t capture all the body language and hesitation that attended so many of the answers to questions. I just want to pick out some bits that I think are worth our attention – in no particular order.

What astounded me more than anything was the staggering ignorance of Robert Wilson on matters of statistics. He was appearing for the British Association of Homeopathic Manufacturers, and was challenged on the evidence that homeopathy actually works. He launched a rambling discourse on sample size in clinical trials, stating categorically that most studies were too small (probably true), and that no study with less that 500 patients could be of any use. He seemed to be totally unaware of how sample sizes actually are calculated, and ignored the other factors such as expected differences, and variability. Only the number of patients mattered to him – one size fits all. This is not an esoteric subject, it is bedrock for anyone involved in life sciences. If he is the champion selected by the serried ranks of homeopathy companies, they can expect to lose their colours pretty quickly.

Wilson appeared in the first session, alongside representatives of the Royal Pharmaceutical Society of Great Britain (RPSGB), Sense About Science, and Boots, plus Dr Ben Goldacre of Bad Science fame. Everybody was asked whether they agreed that homeopathy works. Paul Bennett from Boots had to admit that there was no evidence, but regaled the committee with the mealy-mouthed flannel about customer choice that we have come to expect from his amoral employer. In summary, of the five witnesses, four said there was no evidence – and one of those was happy to carry on making money from gullible customers in the knowledge that they were being misled.

The second session was a bit more interesting. Dr Peter Fisher, director of the Royal London Homeopathic Hospital, was asked by the excellent Dr Evan Harris MP why the water used to prepare a remedy only retained the memory of that tincture and not everything else that had previously been dissolved in it. Fisher declared that the water was purified by double distillation, which removed all previous memories. Now this is an interesting hypothesis, and I am surprised that nobody picked him up on it. He is saying that double distillation resets the memory to zero, a claim that can readily be tested. We just need to prepare a standard remedy and divide it in half. One half we then double distil and use it to prepare a new remedy. We then give each remedy to patients in a clinical trial. There won’t be any difference of course, because neither group of patients will respond. If Fisher knows any other way of differentiating two remedies I’ll be avidly interested.

Another one that slipped through the committee’s net was a curious claim from Robert Mathie, research adviser to the British Homeopathic Association (BHA). He was reacting to criticism of high dilutions beyond Avogadro’s number, stating that many remedies were not so highly diluted and did have some remnants of solute. These had apparently been subject to tests, with positive results. Nobody seemed to remember the so-called `Law of Infinitesimals’, whereby potency increases with higher dilution. Is Mathie now repudiating this? But he got away with it.

One committee member was rather more sympathetic to homeopathy. Ian Stewart MP asked whether there was any evidence that homeopathy did not work, but nobody reacted to the essential silliness of the question. Had I been there, I would have said that I can’t prove that there is not a teapot orbiting the sun between Earth and Mars, but that there is no point in behaving as if there is (with apologies to Bertrand Russell). Such a question, from a member of the Science and Technology Committee, demonstrates a basis misunderstanding of science. The same member asked about evidence for harm from homeopathy, and nobody could provide any concrete examples. I would have mentioned Dr Marisa Viegas who in effect killed a heart failure patient by telling her to stop all drugs and take homeopathy. This was in the context of a question about regulation, and the committee should be told that the GMC only suspended Viegas for a year, and then struck her off because she still thought she was right. Sorry – I can’t give you a link for that case because it’s not on the GMC website (only goes back two years), but I have the minutes of the GMC hearing so email me if you would like it.

There was a lot of discussion about dangerous claims by homeopaths, particularly in relation to malaria prophylaxis. I would have liked to see Mathie pressed a lot harder on why no BHA member has been disciplined for making false claims. Bizarre belief systems always protect themselves from criticism by saying that they don’t support extremism, but they never actually keep their house in order by dealing with their own extremists. The result is of course more extremism.

In general, it surprised me that nobody wondered why we were having this debate about evidence at all. The homeopaths were stating that there is good evidence, and their opponents, led by the redoubtable Edzard Ernst, stated the opposite. We don’t have any such debate about the majority of orthodox drugs. It is usually as clear as the nose on your face whether a drug is effective or not, even though the impact of that on clinical practice might need more clarification. What we have with homeopathy is an elusive spectre of supposed efficacy which evaporates in the cold light of day.

Do look at the written submissions. Mine is there, but frankly the response from sceptics has been pretty pathetic – in terms of numbers not quality. Where were you all? The usual exponents of smokescreen science are there, notably the sociologist Clare Relton who loves splitting hairs about words. I need to steel myself to read Milgrom’s. Interestingly, the Prince’s Foundation takes pretty much the same line as Boots.

Well, I am looking forward to Monday’s session, with Professor Kent Woods of the MHRA, and Mike O’Brien MP and Professor David Harper from the Department of Health. How they defend what they have done with licensing will be an education. I don’t mean what they say (that will be the usual obfuscation), but their facial expressions will be a picture.

Advertisements

15 Responses

  1. Excellent & insightful summary.

    Just for information, it is Evan Harris MP, not Davies.

  2. Great post – very acutely observed.

  3. Mine is there, but frankly the response from sceptics has been pretty pathetic – in terms of numbers not quality. Where were you all?

    Well, not invited… I’d have loved to have contributed something. It seems odd that they took submissions from so many homeopaths, but didn’t contact many from the other side.

    • You didn’t have to be invited – I picked it up from the Randi forum and from Sense About Science. But I agree that if you don’t have the right contacts you might miss therse things. I don’t for a moment think the committee invited homeopaths – the latter just spotted it from media monitoring.

  4. Sorry – forgot / ran out of time – had a frantic last minute dash to pen something for the consultation on regulation of herbalists as well.

    I guess a lot of sceptics will have felt that the scientific case against homeopathy is so clear cut that it is needless for dozens of people to repeat it. Ditto a lot of the sceptical arguemnts, which have been eloquently made by David Colquhoun, Edzard Ernst, Ben Goldacre et al. And of course, almost exactly the same arguments were made three years ago when the MHRA was manoeuvred into taking on the ridiculous “traditional use” thing -see e.g. here, or here.

    One thing that is easier to do, and takes much less time, is to write a brief letter saying

    “As a scientist I agree 100% with the arguments made by Prof XYZ”

    It therefore helps a lot when prominent sceptics like David Colquhoun put their submissions, or those of others like the person who writes as “Allo V Psycho”, up on the web in advance.

    The other difference is that scientists, doctors and academics have about a million jobs to do -so this will always be a hobby (hobby horse?). The homeopaths, on the other hand, are defending their livelihood – and probably have free time too, since in a recession their consultation rates are likely dropping steeply.

  5. Oops – first link was meant to be this one.

  6. […] commentary on the meeting by a clinical scientist summed up Bennett’s contribution […]

  7. Had a quick glance through Lionel Milgrom’s submission and it is appears virtually indistinguishable from his last two (or three?) rants in the Journal of Alternative and Complementary Medicine.

  8. “Ian Stewart MP asked whether there was any evidence that homeopathy did not work, but nobody reacted to the essential silliness of the question.”

    Not silly at all. Far from it. “[All] the millions of experiments confirming modern physics and chemistry also constitute powerful evidence against homeopathy.” –Alan Sokal.

    So powerful in fact that all of the homeopathy RCT evidence is utterly irrelevant and always will be. Doing a homeopathy RCT is doing futile and perhaps even unethical pseudoscience.

    “it surprised me that nobody wondered why we were having this debate about evidence at all. ”

    It didn’t surprise me. It is shocking, but it is at least partly the fault of otherwise good scientists colluding with the homeopaths in their pseudoscientific endeavours: taking their ‘evidence’ seriously – sometimes even doing systematic reviews of it – and failing to give the right answer to the least sillty question they are asked by HoC committees. 😉

    • Yes, I’m familiar with what Sokal says but I don’t think Stewart meant that. He specifically juxtaposed it to the lack of evidence for homeopathy, which in the context meant clinical trials. I think everyone took that to mean, is there clinical trial evidence that disproves homeopathy? It’s related to the `absence of evidence is not evidence of absence’ fallacy. In any case nobody could offer an answer to the question.

      I am aiming the point about having the debate at all, at the witnesses who certainly don’t collude with homeopaths! But maybe the committee will read my blog and ask the question at the next session tomorrow.

      • I’m sure you’re right that Stewart meant it in that sense. Nonetheless, I thought it was a lost opportunity. BTW, I don’t blame anyone in particular for collusion in the pseudoscience of homeopathy – I see it as a collective failure – but I have to say that that is exactly what I think e.g. every systematic review or meta-analysis of homeopathy RCTs I have seen amounts to.

        Interesting – I didn’t know there were more sessions to come. I just hope someone does tell them the /whole/ truth about the evidence concerning homeopathy.

  9. […] Majikthyse gives another interesting recap from the perspective of a clinical scientist. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: