The limited value of ‘statistical significance’ in the real World

Earlier this week I was working from home in the morning. I had the radio on in the background. My normal choice of oral wallpaper is BBC Radio 4. It’s often on, but I’m rarely ‘listening’ to it. I rely on the ‘cocktail party effect’ to pick up on anything vaguely relevant. In other words, listening to the radio for me is a bit like being at a gathering where there is a buzz of conversation in the background. In the main, what we ‘hear’ is filtered out, but if someone where to say something particularly relevant to us, such as our name, then it would tend to attract our attention. So, I am generally deaf to items on such matters as the spotting of a rare bird on a remote Scottish island and the minutiae of fiscal policy here in the UK, but when something about health or science pops up, I can suddenly be all ears.

So, earlier this week my attention was grabbed by an item on the debate about whether the time limit for termination of pregnancy should be dropped from 24 weeks (where it stands now) to 22 or 20 weeks. By the way, this blog article is not ethically, morally or religiously driven, it about science, or rather, the limitations of it.

One side of the argument here states that the abortion time limit should be brought down because babies can (and do) survive when born at an age lower than the current 24-week cut-off. Those opposing the change have generally used the argument that the ‘evidence’ shows that the survivability of infants born very prematurely has not changed in recent years. So, if 24 weeks was good enough when the limit was set, it is good enough now.

The obvious riposte to Gordon Brown’s (and others’) ‘scientifically-based’ argument is that there’s no reason to assume that just because the survivability of very premature infants has not changed, that the abortion time limit right. Maybe we got it ‘wrong’ the first time round and there’s an argument for reviewing the limit.

One individual supporting a review was a woman who was interviewed on Radio 4 who, if I remember correctly, delivered a child at 22 weeks gestation. The child was, she said, left to die. However, because after 36 hours this child had not died, it was duly treated with medical care and survived. According to the mother, while the child was (naturally) a slow-starter, he had caught up and was leading the sort of life you’d expect ‘normal’ children to lead.

It was put to her by the interviewer that infant mortality statistics had not changed, so how could she justify her desire (as if it were not obvious) for the termination time limit to be reduced. What she said, and I’m doing this from memory, was, I think, very telling. She first of all suggested that we need to be a bit careful with statistics. She reiterated the point that children can survive at an age lower than the termination limit. She rounded this off by suggesting that while the statistics may not have changed significantly, for a child who may survive being born very prematurely the issue is very significant indeed.

I think she has a point. And this whole issue reminds me of just how easily we over-rely on the science and statistics. And examples of this, I think, are legion in the medical field.

For example, I have written before about the placebo response and its power in promoting healing. Some (for instance, academics who never go near real patients) dismiss the placebo response as an artefact, and something that is not ‘real’ like the effect you get with, say, a drug that has been ‘proven’ to be effective. My opinion is that if a treatment or approach helps someone, the mechanism behind the improvement is far less important than the fact that they have improved. But I suppose that’s one of the differences between academics and individuals who actually see patients with real problems and who are focused on actually helping people.

Another way science may not be of service of us concerns ‘statistical significance’. This tells us, supposedly, whether there’s some real effect or change going on, or it’s merely something that’s most likely to be due to chance. Statistical significance in scientific studies is denoted by what is known as the P (or probability) value. A value of less than 0.05 is generally regarded as denoting ‘statistical significance’.

Sounds fine so far. Except, I do feel compelled to point out that the choice of 0.05 as a cut-off is utterly arbitrary. It’s a value that the scientific community agree on. It’s a consensus ” it’s not carved in stone like some irrefutable scientific truth. If the scientific community decided that 0.01 was going to be a cut-off, then less things would be ‘statistically significant’. If the limit was set at 0.1 then many more things would be deemed significant. When we understand this, we begin to see just how arbitrary a lot of scientific ‘findings’ really are.

An example of where statistical significance appears to have got in the way of a constructive debate on the subject is vaccination. Our Government here in the UK, most doctors (I suspect) and many commentators would have us believe that vaccination, including the measles, mumps and rubella vaccination (MMR) is ‘safe’. Many will not even entertain the thought that there may be a problem with MMR. They’ll quote the science (some of which is not of the highest quality anyway) in a way that gives the impression, very often, that there is NOTHING AT ALL to worry about.

An analogy may be useful here. Let’s imagine someone decided to do a big study on road safety. Let’s say they counted up the number of times someone, somewhere, crossed the road. And now, let’s imagine, they also count up the number of times someone gets run over (and hurt or killed) as a result of crossing the road. Now, I’m writing this on a plane and can’t even check if these statistics exist. But I think it’s reasonable to assume, that compared to the total number of road crossings, the number of people being knocked down is likely to be very small indeed.

Now imagine we applied some statistical ‘wizardry’ to this (with that arbitrary P value, remember) It’s not too difficult to imagine that one would turn up a result which shows: ‘crossing the road is not associated with a statistically significant increased risk of getting run over.’ Now, many doctors and scientists would interpret this finding as evidence that crossing the road is ‘safe’. However, we all know that while most of the time it is, sometimes it’s not.

Now, getting run over has obvious after effects. Vaccination, on the other hand, may not. The effect, for instance, may be delayed. And also the changes can be more subtle than a broken leg, a ruptured spleen or death. Nevertheless, despite the protestations of some, there is a considerable body of people out there who believe (rightly or wrongly) that their child has been damaged by vaccination. And all too often these individuals are dismissed or patronised.

To get some indication of how some of these parents might feel, imagine for a moment turning up at hospital with your child who has been run over. When you get to casualty the attending doctor asks what happened to your child. You reply that they were run over crossing the road. Now imagine the doctor turns round to you and says with a somewhat withering tone: I don’t think so: Study after study shows that there’s no credible evidence that crossing the road can be harmful to human health.�

Whatever scientists and doctors sometimes contend, the fact remains: accidents can happen.

77 Responses to The limited value of ‘statistical significance’ in the real World

  1. Jayney Goddard 23 May 2008 at 9:47 am #

    As usual – just excellent – thank you John!

  2. Gary Shaw 23 May 2008 at 10:12 am #

    Dear Dr Briffa,

    This is an excellent article that highlights not only the issues of statistics in medicine but also in using them in general. To use a similar example as the babies being born prematurely, they may only have a small chance of survival, but as pointed out, it is literally a matter of life and death so incredibly significant.

    I note here that you didn’t give your personal opinion either way – and whilst this article is excellent it isn’t too telling. Do you personally believe the abortion limit should change?

  3. Peter Killingback 23 May 2008 at 10:15 am #

    re: MMR and other regular vaccinations/immunisations: these are good for the population but not necessarily good for the individual.
    Many practitioners work using population ‘events’, eg. I have a tummy upset, Oh yes there is a lot of this going around, have some ‘X’ that will clear it up! This is on the basis of ‘common ailments occur commonly and uncommon ailments occur rarely’. The really smart practitioner will be very, very aware of the limitations of such approaches.
    Or again as one Dr put it to me, when I consulted about a chest infection, It wont be TB, we’ve had our four cases in this area already this year! To this day I do not know if this was said with tongue in cheek….but I do hope it was.

  4. Brian Abbott. 23 May 2008 at 11:21 am #

    I think the limit for termination should have been set at the time in gestation when it can be proven that a child has survived outside the womb, and has managed,with a initial period of assistance if necessary,to maintain its own breathing, heart and other vital organ functions for a period of time determined by the medical scientific community, with oversite by legal personnel and lay people.The period of time when assistance can be given should also be determined by the medical scientific community, with oversight by legal personnel and lay people.

  5. Dr John Briffa 23 May 2008 at 11:31 am #

    Gary
    This blog post isn’t really about abortion time limits, it’s about the limitations of the scientific method. My personal opinion on the time limits for abortion (which, by the way, is quite uninformed) isn’t really relevant to the real issue at hand, here.

  6. Gordon Taylor 23 May 2008 at 12:26 pm #

    Need more of this kind of scientific questioning. The use of statistical significance is problematic – all it really tells us is that at a 95% significance if we repeated the study we would get comparable results 95% of the time. It doesn’t tell us if the answer is correct, merely that the results are consistent.
    To my way of thinking there is a difference ininterpretation and scientists should be careful to point that out.

  7. Anna 23 May 2008 at 3:32 pm #

    Great topic. I’m always annoyed at the way relative and absolute risk percentages are used in the media. Most people don’t understand those, either. Another post topic?

  8. Alison 24 May 2008 at 8:39 am #

    yes totally agree
    “science”.. statistics.. are black and white
    life is grey, a vague misty kind of grey

  9. irene smart 24 May 2008 at 7:26 pm #

    Thank you for this.. It is not just about abortion of course, it is about what constitues “objective” measures; you have here helped to demonstrate a clear and insightful way to convey the blanket use/misuse of probability in statistics in medicine. While those subcsribing to your blog “get” the misuse of statistics, the road accident scenario made it much more immedate. I shall use this analogy to further the questioning of the status quo by my friends… Neat. Cars, road accidents, ubiquitious understanding. Well, perhaps. Thank you for giving me a suggestion to aid understanding from others.

  10. Angela Howes 24 May 2008 at 8:00 pm #

    I appreciate the valid comments regarding statistical significance but hope that the unfortunate choice of using the mmr vaccine as an example doesnt fire up dissenters again. Consider the ‘significance’ of the rise in infant morbidity and mortality if measles becomes endemic in communities again due to lack of uptake of vaccination.
    Any vaccine has potential to cause harm, however rare.

  11. Dave 25 May 2008 at 5:11 pm #

    The problem with statistical significance tests is that they are answering a different question than what is often assumed. Experiments are usually done to test some hypothesis. Statistical significance essentially answers the question “What is the probability that I would have observed this data GIVEN that a particular hypothesis is true”. In other words, it tells you how well the hypothesis supports the data, not how well the data supports the hypothesis. The proper question is is “What is the probability that the hypothesis is true given that I observed this data” (and you also have to include effects of other background information, such as prior experiments). Not only does this answer the right question, it allows you to do things like compare competing hypotheses. For example, one might find that the lipid hypothesis of CHD is supported at the 90% level; but that doesn’t make it right, just 90% certain. If you test an alternative hypothesis such as “Refined carbohydrates cause CHD” and find it supported at the 98% level, then it’s obviously more desirable (I’m oversimplifying the process, but that’s the basic idea).

    Annoyingly, nearly all scientific experiments are interpreted in this backward manner, despite there being a rigorously derived mathematical framework (called Probability Theory) to do it the right way. Edwin Jaynes is the modern father of Probability Theory, and has an excellent (albeit mathematically dense) book on the topic, called Probability Theory: The Logic of Science. It’s not an easy read, but worthwhile.

  12. Liz 25 May 2008 at 5:55 pm #

    Your analysis of the statistics reminds me of the American weather reports where they say things like “There’s a 30% chance of rain”.

    Whatever the percentage, if it rains you get wet.

    Which shows the limitations of statistics.

  13. Dave 25 May 2008 at 6:44 pm #

    The real power of the Probability Theory approach is that when combined with Information Theory allows you to do logical reasoning in the face incomplete information. The limitations of what you know become clear, especially when trying to make decisions. This is important in a number of places, not the least of which making medical treatment decisions.

    “30% chance of rain” simply tells you the degree of belief about a future outcome. How that affects your decisions has to do with how much you care about whether or not you get wet.

  14. Derrik 27 May 2008 at 11:40 pm #

    Sometimes people are right for the wrong reason, sometimes they are wrong for the right reason, you have managed to be wrong for the wrong reason.

    Never mind, try again.

    Oh I’ve just realised you have a book published, ghost writen was it?

  15. Lex 28 May 2008 at 12:24 am #

    Hi Dr Briffa,

    I am intrigued by your criticism of academics dismissal of the placebo effect in medical trials. I would be surprised if any academics really question the efficacy of the placebo effect in certain cases as it has been well documented.

    It is quite obvious that handing out sugar pills to patients instead of pills containing active ingredients (and therefore inevitably side effects) is safer and cheaper in these cases. The reason this is not done in practice is ethics rather than science. It is not considered ethical to give a patient a pill and lie to them about its content. Whether these ethical guidelines are correct or not is a grey area, but it s not the issue here.

    The reason scientist are careful to take into account the placebo effect in medical trials is because they want to specifically test the active ingredients within the drugs. To test the drugs without also testing the placebo effect through double-blind trials would not allow the efficacy of the drug to be measure.

    ‘Big pharma’ (and the homeopathy industry, not to mention the food supplement industry) would love to be able to include the placebo effect when reporting the results of their latest drugs (when it comes to homeopathy and food supplements they always do include the placebo effect- they would run a mile from a double-blind trial). Drugs with active ingredients which work for one specific illness, for example to counteract hyperactivity, could then be sold for a wide range of illnesses, for example depression, when in actual fact, for depression any placebo would work as well. Bigger market results in bigger profit. Who cares about the side-effects. (Fortunately the only side effect of homeopathy is a significantly reduced bank account!)

    Your argument for this to be allowed is based on the idea “well so what? If the placebo effect helps why not?”. The simple answer to this is two fold: sugar pills are cheaper and much less dangerous than prescribing drugs whose active ingredients are not designed for the illness at hand. If it is unethical to give out sugar pills pretending they are real drugs, surely it is unethical to give out real drugs which only work through the placebo effect. If you think it is ethical to give out sugar pills, then give out sugar pills when it will help, again, they are far cheaper and much safer than using actual drugs.

    Hope this clears up the reason why academics are so obsessed with eliminating placebo effect when testing drugs, and why alternative or complementary medicine providers aren’t.

  16. Dr John Briffa 28 May 2008 at 8:00 am #

    Lex
    I entirely understand the need of some people to perform randomised, placebo controlled trials (chiefly, in an effort to discern whether what is being tested has a ‘real’ effect or not). However, in the real world (that’s real people, with real problems) the fact that the placebo response may account for a lot of even the whole of a clinical response is not generally important for those the treatment is intended to help (those real people with real problems, again).

    Whatever your views regarding the ethics of giving placebos the fact remains that some doctors do give them, and they may benefit to individuals too. I don’t advocate the use of placebos (as you seem to suggest), but the fact that they are used in medical practice is why I have written about this in the past. Also, remember that some placebos are self-administered, so the ‘ethics’ of prescribing them does not come into it.

    Also, even if study has found a treatment to be no better than placebo, we actually do not know (for certain) if any benefits gained from this treatment in an individual were due to merely a placebo response, or some real effect, or both. I’d be very wary indeed about extrapolating from science to the individual with the level of certainty many doctors and academics often exhibit.

    And all of this is a diversion anyway, seeing as the post was not about the placebo response, but mainly about the relevance of ‘statistical significance’ in the real world. So, whether a treatment is deemed more effective than placebo is determined by the application of arbitrarily set criteria. Set other criteria, and we get a different ‘result’. Now, suddenly, the world of science-ology does not seem as objective as many scientists and academics would have us believe it to be.

    You tell us that ‘they’ (whoever ‘they’ are) in homeopathy and food supplements would ‘run a mile from a double-blind trial’. I can’t speak for homeopathy (as I don’t know much about it), but you are simply incorrect with regard to double-blind studies on nutritional agents. There is, in fact, quite a body of evidence regarding the effects of nutritional agents on health that is double-blind in nature. Here’s a recent study as an example: Su KP, et al. Omega-3 Fatty Acids for Major Depressive Disorder During Pregnancy: Results From a Randomized, Double-Blind, Placebo-Controlled Trial. J Clin Psychiatry, 2008 Mar 18 [Epub ahead of print].

    Now, either you didn’t look for the presence of such research or you just assumed it did not exist. Or maybe, you looked, found there was some evidence and decided just to ignore it all the same. It’s just this sort of subjectivity (and apparent bias and prejudice) in science that I think needs to be exposed.

    If you have any views on the arbitrary nature of ‘statistical significance’ I’d be pleased to hear them….

  17. Dr John Briffa 28 May 2008 at 8:10 am #

    Derrik
    Oh do please tell us why I’ve got it wrong for the wrong reason, or are we just to take your word for it?
    And what’s that too – some wild speculation about whether I write my own books for not (like that’s got anything to do with it anyway….).
    Indulge me in some idle speculation of my own. I see from the information that comes with the IP address of your computer that you have a connection with one of England’s ‘finest’ seats of learning. Maybe you are an academic yourself?

    Perhaps you could tell us? And while you’re at it, why not reveal your identity, and let us all see who it is that makes assertions that he/she does not feel the need to substantiate in any way whatsoever….

  18. jdc 28 May 2008 at 3:17 pm #

    Dr Briffa,

    I read your post with interest, after my attention had been drawn to it by another blogger, and I thought I’d ask a couple of questions of you / share a couple of views with you.

    Of your section on statistical significance and p-values, one person has pointed out: ‘that’s all true, although the fact that p<0.05 is a totally arbitrary choice isn’t exactly a secret. We all know it. Often, people will demand p<0.001. That’s why we quote p values rather than just printing “yes” or “no”’.
    Having read the two blog posts, it seems to me that you are trying to imply (unfairly) that, because the usual p-value of 0.05 was reached by consensus, p-values and statistical significance are somehow meaningless and scientific findings themselves are therefore arbitrary. This seems like the perfect excuse for someone who wishes to recommend treatments, supplements or diets that are not backed by scientific evidence. Also, your post makes it seem as if scientists are somehow slapdash when it comes to use of statistical methods – like the scientists have shrugged their shoulders and said ‘p-value? let’s just make it 0.05′. You ignore the efforts made by scientists to take account of certain factors when completing research – one example being the use of Bonferroni correction when testing more than one hypothesis on a set of data.

    Unfortunately, a policy of ignoring scientific evidence leads us down a somewhat depressing path – we go from having a ‘workable-but-imperfect’ system for discovering truth to… having no system at all. Without having the scientific method available for us to use, surely any one anecdote is as good as any other anecdote? And any single anecdote may be used to justify, well ” just about anything.

    Worryingly, you show us some examples of this approach elsewhere in your blog post. You seem to imply that the experience of one mother, rather than a properly conducted study such as Epicure, should inform abortion policy in this country. You then go on to deny that the MMR vaccine is safe, and use an analogy about road accidents in order to make your point. While it is theoretically possible that the vaccine may cause autism, there is currently no good reason to think that it does. There are real risks with vaccines – such as the possibility of an allergic reaction – and those groups considered to be most at risk from vaccines are advised not to get jabbed. For more on the risks of the MMR vaccine and the risks of the diseases the vaccine prevents, try http://www.mmrthefacts.nhs.uk/library/sideeffects.php – frankly, the “MMR – The facts” page will be of far more use to parents worried about MMR than anything you have written on the subject Dr Briffa.

    jdc

  19. jdc 28 May 2008 at 3:52 pm #

    Just one more thing: “You tell us that ‘they’ (whoever ‘they’ are) in homeopathy and food supplements would ‘run a mile from a double-blind trial’. I can’t speak for homeopathy (as I don’t know much about it), but you are simply incorrect with regard to double-blind studies on nutritional agents.”
    Mmm. You’ve cited just the one double-blind study to refute the accusation that the supplement industry would ‘run a mile’ from a double-blind trial. I still think that the accusation stands. Most food supplement companies don’t conduct scientific trials of their products at all – and why should they? After all, they aren’t required by regulation to do so and if they took the chance and conducted a study, then the results might well be ‘not to their liking’ and the money they have spent on this scientific study would be considered (at least by the finance dept!) to have been wasted. Other supplement companies give their product to schools in order to promote this initiative as a ‘trial’. Still others are stated to have approached researchers but insisted on retaining control of the data. What ethical scientist could abide by such terms? So, while there may well be some double-blind studies of nutrients, your commenter Lex still makes an excellent point regarding the general state of affairs in terms of supplement companies conducting proper research. As for homeopathy firms, I’ve seen some info on Boiron and apparently they spend 18.5 times as much on marketing as they do on R&D.

    Of course, these tricks aren’t exclusive to homeopathic and food supplement companies – ‘Big Pharma’ spends twice as much on marketing as on R&D and has form when it comes to burying studies or cherry-picking the ones they want to submit. But let’s leave the readers of this blog under no illusions – homeopathy firms and food supplement companies are in business and they act like businesses in protecting themselves rather than the consumer. Protection of the consumer is generally undertaken by government and by executive agencies of the government. We have to rely on the MHRA, ASA, FSA and Trading Standards to protect us because Big Pharma, Homeopathy and Nutritionism won’t.

    jdc

  20. Dr John Briffa 29 May 2008 at 11:28 am #

    jdc – in response to post no. 18

    You’re right, the arbitrary nature of p-values is not a ‘secret’, but you’d be surprised just how much this fact is not fully understood or appreciated by those with no scientific training (like the majority of the readers of my site, I suspect). So, nothing wrong with pointing that out, I reckon.

    I didn’t state not even imply that scientists are ‘slapdash’ in the application of p-values, merely that what p-value is chosen is arbitrary in nature, which means that what is viewed as ‘significant’ is also quite arbitrary.

    What I think happens in the real world is that when a scientific study pronounces a finding that is said to be ‘statistically significant’ or not, is that people interpret that to mean that a drug works or doesn’t or a vaccine is safe or not. And what I’m saying (if this wasn’t absolutely clear in the post) is it’s not like that: because the cut-off for what we determine to be ‘significant’ is arbitrarily set. This may be obvious to you. But as I pointed out above, I actually don’t think it’s obvious to everyone.

    The road accident injury analogy was used in an attempt to provide a graphic example of how science and P-values may pronounce something to be safe – safe, perhaps from the standpoint of an arbitrarily set criterion, but not safe and possibly deadly for the person who gets run over.

    You also appear to misrepresent me in suggesting that I have the opinion that science has no value. I don’t hold that view at all, and you would know if you spent just a few minutes trawling my site: it regularly cites scientific evidence.

    However, science has considerable limitations (something that some scientists and academics are loathe to admit, it seems), and we must be aware of these limitations if we are to interpret science properly. So, for the record, I support the concept of science, but I’m no slave to it.

    And I also know that one’s experience in practice (in the real world, with real people with their real problems) is important too. It’s not just me that thinks this: evidence-based medicine is described in a seminal editorial of the subject in the BMJ as: ‘…about integrating individual clinical expertise and the best external evidence.’ See: http://www.bmj.com/cgi/content/full/312/7023/71

    So, jdc, perhaps you’d like to share with us some of you clinical expertise. Or is healthcare, for you, an essentially ‘academic’ pursuit? Perhaps you can tell us….

    And so to MMR…

    First of all, you suggest that I believe MMR to be unsafe. Actually, I said no such thing. My point is, we can’t be sure that it is safe. Those two positions are not the same.

    Anyway, on to the ‘evidence’ that you refer to that we should, apparently, take comfort in. In the link you supplied under ‘How do we know that MMR is safe?’, we are informed that:

    ….there is a great deal of evidence to suggest that the vaccine is safe.

    * The MMR vaccine is used in over 90 countries, including the whole of the European Union, Australia, New Zealand and the USA

    * Over 500 million doses have already been given worldwide

    * In the USA, the MMR vaccine has been given to children for nearly 30 years

    * Long term research conducted in Finland has reported that no deaths or permanent damage has ever been linked to the MMR vaccine

    * The World Health Organization (WHO) describes the MMR vaccine as a ‘highly effective vaccine which has an outstanding safety record’

    There is a risk that if large numbers of children do not have MMR, the diseases the vaccine prevents will come back.

    Let’s have a look at this ‘cast iron’ advice in just a little depth:

    The first three bullet points tell us how widely and for how long it has been used (this is no different from saying ‘billions of people have crossed roads over the past 50 years’ ” it tells us NOTHING AT ALL about safety – NOTHING).

    One other bullet point refers to long-term research in Finland, but does not reference this, for some reason. I suspect what is being referred to here is the research that formed the basis for the following letter:

    Peltola H, et al. No evidence for measles, mumps, and rubella vaccine-associated inflammatory bowel disease or autism in a 14-year prospective study. Lancet 351(9112);1327-8. This letter was based on data that came from a previous study by the same team: Peltola H, et al. The elimination of indigenous measles, mumps, and rubella from Finland by a 12-year, two-dose vaccination program. N Engl J Med. 1994 331(21):1397-402.

    This study seems to have been used as evidence for a lack of link between MMR and autism because it apparently showed NO cases of autism after millions of MMR administrations. The devil of course, is in the detail: autism cases were NOT MONITORED as part of this. Some other adverse effects were monitored, it seems, but not autism (for some reason). In fact, in the whole of this study, the words ‘autism’ appears precisely NO times.

    It is quite shocking to me (and perhaps some other people reading this) that this evidence has used by some to conclude that MMR does not cause autism, when it is completely inadequate from a scientific standpoint. Some would say it is actually ‘fraudulent’ to use this science in an effort to ‘persuade’ the unsuspecting public that MMR does not cause autism.

    So, let’s just hope that the WHO is not relying on this sort of ‘evidence’ when it pronounces MMR to have an ‘outstanding safety record’ (though I very much suspect it is).

    And then all this is rounded off with the usual clarion call: “There is a risk that if large numbers of children do not have MMR, the diseases the vaccine prevents will come back.” Though quite what that has to do with the safety of MMR is anyone’s guess.

    So, jdc, you have drawn our attention to this ‘MMR ” the facts’ site, and now perhaps you’d also like to comment on the robustness of the ‘evidence’ this site uses to pronounce MMR as safe, specifically with regard to autism (by the way, it was you that raised the autism issue, not I). Please do tell us what you think of the Peltola study in particular (the one which didn’t even gather data on autism). This is an honest request, please do let us know your views.

    You say there is no ‘evidence’ that MMR causes autism. Well, I don’t know if MMR can cause autism or not. But, personally, I am loathe to dismiss the ‘evidence’ that comes in the form of countless parents who say that their child was developing normally, until they had MMR vaccination shortly after which they regressed into an autistic state.

    And one other thing that may interest you (or other readers of this) is that the US Government recently conceded (out of court) that a child’s (Hannah Poling) autistic state had been significantly contributed to by vaccines she had as a toddler. This child actually had 5 vaccinations (a total of 9 vaccine components) in a single day. However, it’s not too difficult to imagine how a smaller vaccine load could still lead to problems in susceptible children.

  21. Dr John Briffa 29 May 2008 at 11:38 am #

    jdc – in response to post no. 19

    Lex wrote ‘they would run a mile from a double-blind trial’. This is stated in absolute terms, no? Lex did not use words such as ‘generally’, or ‘usually’ or ‘tend to’. No, they ” all of them ” would not engage in double-blind research is the assertion.

    Now, as I said, there is a significant body of double-blind research in the area (some of which is industry-funded, of course). I actually started out with a list of studies to rebut Lex’s claim. But then I remembered something I think Karl Popper said about swans: If the hypothesis is that there are only white swans, I only need to show the existence of one black swan (not two, or 10 or 100) to disprove the hypothesis.

    And then I wondered if someone with a very pro-science stance would ‘take the bait’, focus on the fact that I only presented one study, and in so doing would betray ‘good science’ (and even logic) and demonstrate the subjectivity and prejudice that is so often found in science and academia.

  22. Elizabeth 29 May 2008 at 6:15 pm #

    In partial reply to jdc’s various comments:- the website known as “mmrthefacts” is regarded by parents of “allegedly” vaccine-damaged autistic children with considerable disdain. At one point it recommended that a child who had reacted badly to the first MMR should be given the so-called booster MMR regardless of its prior adverse vaccination reaction. Surely this goes against commonsense as well as good medical practice?

    The example I always use is that penicillin has been a great boon to mankind but my doctor has warned me that the next time I receive it could prove fatal (he wasn’t joking). My Medicalert medallion is in my purse.

    The attitude “one size fits all” fails to take any notice of individual susceptibilities (see previous paragraph) and there is considerable evidence from the parents of autistic children that autoimmune disorders, allergies and gastrointestinal disorders appear with alarming frequency in the family medical histories of children who’ve descended into regressive autism following vaccination. Why isn’t this being investigated?

    One thing that really worries me is the lack of intellectual curiosity in the orthodox medical profession – they’re being told about adverse vaccination reactions by parents of autistic children and yet no-one seems to think this should be investigated. I think we’re all aware that very few adverse vaccination reactions are notified via the “yellow card” system. Where are the U.K. research studies involving clinical examinations of allegedly vaccine-damaged children?

    On a personal note I can inform jdc that, having had boy/girl twins and watching my normally developing daughter descend into regressive* autism immediately following MMR, there can be no doubt that something happened. The point when doctors go quiet and then hurriedly change the subject is when I point this out and then say “we’ve got dated camcorder footage”.

    * Loss of eye contact, total loss of developing speech (mute for two years), appeared not to understand what was being said to her, etc. etc.

  23. Occam 30 May 2008 at 7:07 am #

    Your comments about 95% confidence limits and the analogy to road crossing casualties are not correct I’m afraid .
    Statistics and the commonly accepted p=0.05 criterion is used to demonstrate CAUSE & EFFECT, not absolute numbers or relative risk ie the chance of being run over in your example, which is what you are talking about in your comment if you get run over then for sure its 100% as far as you are concerned.

    95% confidence limits are an arbitary limit agreed, but its a bench mark, (although you can use p=0.1, but obviously with a lower level of confidence that the hypothesis is correct)
    So you use confidence limits, in your analogy of roads to demonstrate the effect of different road surfaces, lighting, speed limits etc etc & determine which was most effective at reducing casualties.
    Your last comment refers to incidence & risk, thewhole MMR/autism debate is around cause & effect and epidemiological studies on vaccinated v unvaccinated populations or if the autism rate dropped when vaccination policy changed.

  24. Dr John Briffa 30 May 2008 at 7:39 am #

    Occam
    Yes, you can use probability values to assess different road conditions to, as you suggest, assess which is the most effective at reducing casualties, but you can also use them when assessing what effect ‘crossing roads’ or ‘not crossing roads’ has on risk of casualty, and if one of these factors carries a ‘statistically significant’ enhanced risk of casualty compared to the other.

    With regard to MMR, you refer to epidemiological studies. My point is that these are simply inadequate for ‘proving’ that MMR is ‘safe’, and also tell us nothing about ‘cause and effect’. So, thank you for drawing our attention again to this.

  25. Occam 30 May 2008 at 8:59 am #

    Your comment on crossing roads says “assess the risk” exactely my point of course, you can calculate a rik of doing this and of course there will be a highly significant difference statistically in the risk of death if you cross the road compared to not crossing the road, so what’s your point?

    MMR unfortunately as you will know you cannot prove a negative, ie that there is not any harm, but you can test whether there is harm statistically between the two populations, that’s the whole point testing the null hypothesis that there is a difference (ie harm) at the p=0.05 level. I stand to be corrected but I believe that this has not been demonstrated in epidemiological studies of MMR vaccination

  26. Dr John Briffa 30 May 2008 at 9:56 am #

    Occam – in response to comment no. 25

    You state that: ‘of course there will be a highly significant difference statistically in the risk of death when you cross the road compared to not crossing the road.’ Can I ask how you KNOW this? What about if people are run over and killed while standing on or walking along the pavement? What about people suffering ‘death’ from other causes?

    I suggest that your assumption is a bit hasty, and also that it’s not very ‘scientific’ to predict with certainty the results of studies that haven’t even been done (as you have done). But this isn’t the first time a commenter here who is a congregant of the church of science-ology has exhibited some distinctly unscientific thinking (see comments 19 and 21 for another example).

    The reality is that it is entirely possible that a ‘scientific’ study will find no statistically significant enhanced risk of death with crossing the road compared to not crossing the road. And studies that find MMR vaccination is not ‘statistically significantly’ associated with some adverse effect do not prove that MMR cannot cause that adverse effect. THAT’S my point.

    And with regard to proof of some problem (e.g. autism) with MMR, you’re obviously right: you can’t prove a negative. But you wouldn’t necessarily know that from the people who have used epidemiological evidence to insist that MMR does not cause autism. Again, I feel compelled to thank you again for drawing our attention to the inadequacy of the evidence used to ‘persuade’ us MMR is safe.

    Your comments here serve quite elegantly, I think, to highlight some of the inadequacies of the scientific method (which was the aim of the original post), not to mention the distinctly unscientific approach some take in the name of science.

  27. Occam 30 May 2008 at 10:16 am #

    You’re wriggling, you would control & test that there was no significant difference from other causes of death, ie knocked down on the pavement between the control & experimental groups, the factor being tested for is the act of crossing the road, a properly controlled study, as in clinical trials, would ensure there were not confounding issues, look at the published epidemiological breakdown by race, age, sex etc etc in clinical trial reports comparing test and comparator drugs.

    Again you wriggle re autism the conclusion has been drawn that there is no evidence that there is a causal relationship between MMR & autism, ie the null hypothesis that there is harm was not demonstrated.

  28. Dr John Briffa 30 May 2008 at 10:35 am #

    Occam – in response to comment no. 27

    Fine by all means control as much as you like, but you didn’t answer the question: How is it that you managed to predict with certainty the result of a study that hasn’t been done? I’m hoping the irony of you accusing ME of ‘wriggling’ is not lost on you, though I suspect it might be.

    And with regard to MMR, please do tell us all how it that I’m ‘wriggling’ here: As I have stated quite clearly, my position is not that MMR causes autism, only that it has not been proven NOT to cause autism.

    And another little ‘scientific’ point that seems perhaps to have passed you by: the epidemiological evidence use to vindicate MMR could never prove (or disprove) a CAUSAL relationship between MMR and autism. But I suppose that doesn’t matter much to those who have made their mind up before the ‘facts’ are in (you know, those distinctly unscientific scientists I referred to in my last comment…)

  29. cynic 30 May 2008 at 11:55 am #

    Dr B,

    Your road-crossing analogy has got a lot of people’s backs up. I feel I must correct one implicit point:
    Significance testing if performed correctly* has nothing to do with how rare an event is. This was a bad point to start on and it has distracted from your essential and correct point that even rare events can happen and their consequences can be devastating to those involved.

    We have lots of roads and sadly lots of road accidents so there will no doubt be lots of data. I actually think that a well-performed* study would find that crossing the road WOULD be associated with a statistically significant increased risk of getting run over (at whatever level of confidence/chosen p-value).

    And by the way I obviously cannot use that information to assess what anyone in particular chances of getting across a road intact actually are. I don’t think anyone would disagree with that. (I don’t think you and Occam actually disagree).

    *It’s a big “if”. In short p-values only work properly if studies have enough data points that the rare event can kinda be spotted a few times and separated out from randomness if you like (you yourself have pointed studies out where this hasn’t occurred before I believe). The statistics around low number studies get very weird indeed.

  30. Dr John Briffa 30 May 2008 at 12:34 pm #

    Cynic
    You say: “In short p-values only work properly if studies have enough data points that the rare event can kinda be spotted a few times and separated out from randomness if you like..”

    But what if study isn’t large enough (i.e. there are not enough data points) and/or something is too rare to ‘spot’, what then?

    You (as did Occam) maintain that a study on road crossing and death/injury/whatever would produce a statistically significant result. And in so doing you seem to have, like Occam, developed magical clairvoyant abilities. It’s a neat trick, but not very scientific, if I may say (do excuse me if I’m repeating myself).

    Now, even if the analogy of road-crossing doesn’t work for some and has even ‘got some people’s backs up’, can I ask why that’s REALLY important? Because, before we allow ourselves to stray too far from the real issue at hand, the point is that the epidemiological studies used to ‘prove’ MMR does not cause autism have done no such thing. And never will they. I don’t dismiss epidemiological evidence out of hand, but as we all should know, they can never be used to prove (or disprove) that MMR can cause autism.

  31. Peter Killingback 30 May 2008 at 1:37 pm #

    If only we had had some PROBABILITY STUDIES on WMD and the time it takes to load (from storage..think what this really means) them, perhaps we would not be taxed quite so heavily now!

    My initial comment still stands: immunisation/vaccination is good for the population, but not necesaarily good for the individual. Thus with diphtheria, more of the population might die of infection without immunisation than would die if the population was immunised. Single blind time dependent population studies do demonstrate that this is the case; but from what others have told me, I’m glad I havent had a damaged child from diphtheria immunisation.

    RE: statistics etc.discussed;we all need to remember that stats profile a POPULATION, not individuals. and the Pvalues are probabiities NOT necessarily certainties!!

    THE POPULATION needs more of these discussions to try and get more people to understand what “the figures” mean. If I could make it really interesting, I would teach it in schools and it would be on the GCSE maths syllabus – and before someone says it is taught, It would seem from these discussions that not many understand it; and then we have the conudrum,badly taught or mental inability to understand?

  32. Dr John Briffa 30 May 2008 at 1:59 pm #

    Peter
    If the flimsy, unscientific, biased and sometimes irrelevant arguments put forward here (by people I think would describe themselves as ‘scientists’) are anything to go by, I suggest that the issue of scientific illiteracy is mainly a problem of how science is taught, and not so much an issue of comprehension.

    I wish the aims of those in the field were as laudable as yours, I really do. Because I reckon that not everyone in the field of science actually wants members of the public to understand science, because if they did, it would be a whole lot harder to ‘persuade’ the public of things that have no basis in truth or fact, but are rooted in some ideological, political or commercial agenda.

  33. cynic 30 May 2008 at 2:45 pm #

    If there’s not enough data points then surely, we simply cannot say either way. The scientific thing to do, you must agree, would be to investigate further and seek out more data?
    Or how else can we hope to move forward from that point?

    I am not familiar with but imagine evidence of MMR/autism link is pretty complicated and not clear-cut at all. However in my view your overly-simplified and hyperbolic road-crossing example does no one any favours. Within this example though you have also definitely given the impression that if something is a rare event then any kind of statistical tools used to examine it cannot be trusted. This technical point is not true and was all I was seeking to correct. I notice you did not engage with that point at all.

    We should of course always be very cautious when using statistics. They can be very useful if used properly so maybe we should try and encourage better use of them rather than making them the bad guy?

    I of course agree that epidemiological evidence cannot prove or disprove any causal relationships.

  34. Dr John Briffa 30 May 2008 at 3:12 pm #

    Cynic

    “If there’s not enough data points then surely, we simply cannot say either way. The scientific thing to do, you must agree, would be to investigate further and seek out more data?”

    Yes, I agree entirely. I genuinely would like to credit you for your reasoned, reasonable and ‘scientitfic’ (truly) approach. And in the case of MMR and autism it seems to me that further study is needed. What I’d like to see less of, personally, is scientists and politicians insisting and bullying people into believing something that they do not know to be true.

    “Within this example though you have also definitely given the impression that if something is a rare event then any kind of statistical tools used to examine it cannot be trusted. This technical point is not true and was all I was seeking to correct. I notice you did not engage with that point at all.”

    I didn’t state or even imply, I think, that NO type of statistical tool can be ‘trusted’. The original post was an attempt to explore the limitation of ‘statistical significance’, and its arbitrary nature. We can trust statistics if we want (I refer to them regularly on this site…), but I do think we should be upfront and honest about their limitations too.

    “I of course agree that epidemiological evidence cannot prove or disprove any causal relationships.”

    Agreed. Now tell that to the scientists who cite epidemiological evidence as ‘proof’ that MMR does not cause autism.

  35. Anthony 30 May 2008 at 3:25 pm #

    Now tell that to the scientists who cite epidemiological evidence as ‘proof’ that MMR does not cause autism.

    Epidemiological evidence is not the only evidence that has shown no evidence for an association between MMR vaccine and autism. There are virological studies which also undermine the hypothesis, as well as testimony from one of Wakefield’s team that Wakefield’s study suffered from false positives (which he allegedly knew about).

    I could claim that diabetes is caused by MMR vaccine, and would be able to defend my claim on the same basis you defend the autism-MMR vaccine hypothesis. i.e. You have no evidence disproving my claim.

    Of course, the careful reader will notice that I gave no evidence for my claim, which is effectively what Wakefield did at his press conference.

  36. Dr John Briffa 30 May 2008 at 3:45 pm #

    Anthony

    “Epidemiological evidence is not the only evidence that has shown no evidence for an association between MMR vaccine and autism. There are virological studies which also undermine the hypothesis, as well as testimony from one of Wakefield’s team that Wakefield’s study suffered from false positives (which he allegedly knew about).”

    I note you suggest this evidence ‘undermines’ the hypothesis, but you seem to have stopped short of saying it ‘disproves’ it. Perhaps you can clarify?

    Also, who said anything about Andrew Wakefield? Specifically, who says it’s only his putative mechanism that could be behind the putative link between MMR and autism? Maybe some other mechanism is at play, no? Let’s be methodical about this, I suggest, and not limit ourselves to one possibility.

    So, even if there is evidence that ‘undermines’ Andrew Wakefield’s theory, that in no way disproves the hypothesis that ‘MMR can cause autism’.

    “I could claim that diabetes is caused by MMR vaccine, and would be able to defend my claim on the same basis you defend the autism-MMR vaccine hypothesis. i.e. You have no evidence disproving my claim.”

    Agreed, except where, Anthony, does it look to you that I claimed that ‘autism is caused by the MMR vaccine’? Or are you referring to someone else’s assertion?

  37. Peter Killingback 30 May 2008 at 4:15 pm #

    There is another aspect to NMR immunisation that I have never seen/heard anyone mention. And that is the adjuvanrt effect of one “active principle” on the other two “actrive principles”. For this reason alone, if I had to have children immunised I would insist on separate N,M and R and they would given separated by about 14days, by which time immediate imflammatory reactions would have subsided.

  38. MG 30 May 2008 at 4:34 pm #

    Elizabeth, I think your point about lack of intellectual curiosity about MMR is underlined by Angela Howe’s comment: “I hope that the unfortunate choice of using the mmr vaccine as an example doesnt fire up dissenters again.”
    If anyone raises doubts about the safety of the MMR vaccine they are dismissed as irresponsible for expressing views that might dissuade parents from giving the vaccine, and therefore putting lives at risk. If the medical profession were so sure of their ground they would welcome further research in this area .

  39. Anthony 30 May 2008 at 5:20 pm #

    Agreed, except where, Anthony, does it look to you that I claimed that ‘autism is caused by the MMR vaccine’? Or are you referring to someone else’s assertion?

    I was referring to Wakefield’s assertions, repeated by others. And which you have a post about entitled “Why the MMR-autism ‘war’ is far from over” and state:

    It seems to me that some distinctly shoddy science and no small amount of bullying has been used in an attempt to ‘silence’ those who dare suggest there is a link between MMR vaccination and autism, including countless parents who believe they witnessed the regression into autism of their children after MMR vaccination before their very eyes.

    Distinctly shoddy science?

    Having your cake and eating it is a neat trick, which few succeed at.

  40. Dr John Briffa 30 May 2008 at 5:58 pm #

    Anthony
    “I was referring to Wakefield’s assertions, repeated by others. And which you have a post about entitled “Why the MMR-autism ‘war’ is far from over”…

    Let me see, does that mean because I write about someone that I must automatically share their views and beliefs? Or are you simply going to ‘autoresponse’ mode and suggesting that anyone who questions the safety of MMR (me) believes MMR is unsafe. Where is the logic in either of these stances? Or maybe you had something else in mind. Please do clarify.

    “Distinctly shoddy science?
    Having your cake and eating it is a neat trick, which few succeed at.”

    I have claimed that the science used to claim or ‘prove’ MMR causes autism is shoddy. If you’d like to refute that, go ahead.

    But please do tell us Anthony, how is it that I’m having my cake and eating it? That’s a genuine question, so please don’t ignore it (see below)?

    A couple more questions if I may:

    In a previous comment I asked:

    “I note you suggest this evidence ‘undermines’ the hypothesis, but you seem to have stopped short of saying it ‘disproves’ it. Perhaps you can clarify?”

    You did not reply. Why, and would you care to do so now?

    I also suggested that if whatever evidence you have ‘undermines’ Andrew Wakefield’s theory, that in no way disproves the hypothesis that ‘MMR can cause autism’. I thought you’d like to engage with this point but you haven’t. I’m asking you to do so now.

  41. Andrew 30 May 2008 at 6:03 pm #

    Agreed. Now tell that to the scientists who cite epidemiological evidence as ‘proof’ that MMR does not cause autism.

    I don’t think anyone really does that — I’ve never seen anyone take that stance. It’s something of a straw-man. Sure I imagine a few people do that, but that’s not the main pro-MMR viewpoint, and arguing against those people without acknowledging that most pro- as well as anti-MMR groups both disagree with them is reckless and irresponsible in a debate where people are actually dying of preventable diseases because they’ve been lied to and told that the prevention causes autism.

  42. Dr John Briffa 30 May 2008 at 6:37 pm #

    Andrew
    You’re joking, right? Check out the UK Government’s very own summary of the ‘evidence’ that MMR is safe that jdc very kindly drew our attention to in comment 18 that I presented (along with a critique) in comment number 20: 5 bullets points, one refers to ‘science’ (if you can all it that) and it was entirely epidemiological in nature. Now, do you think the job of writing the ‘How do we know MMR is safe’ page of the “MMR ” the facts’ website for the Government was given to scientists, (or perhaps just farmed out to the teaboy)?

    And that’s a joke about me being ‘reckless and irresponsible’, no? Tell me how it’s reckless and irresponsible to point out that we don’t know if MMR causes autism, and perhaps have the temerity to suggest more work needs to be done. Imagine, if you will, a scenario where it turns out that MMR can cause autism. Do you think it will be viewed that our politicians and some scientists acted responsibly in dragging their heels on this, constantly insisting that MMR does not cause autism despite this not being established as fact? And do you think that those who called and pressed for appropriate research to be done will be labelled ‘reckless’? It’s a strange old World some of you scientists seem to inhabit, I reckon.

  43. Dr John Briffa 30 May 2008 at 6:56 pm #

    Anthony and Andrew
    I forgot to mention something: While I clearly don’t see eye-to-eye with you, I do genuinely respect your willingness to be transparent regarding to your identities. Personally, I’d like to see a bit more of that here and elsewhere.

  44. Anthony 30 May 2008 at 10:40 pm #

    John,

    I have claimed that the science used to claim or ‘prove’ MMR causes autism is shoddy. If you’d like to refute that, go ahead.

    Let’s set out your original comment:

    It seems to me that some distinctly shoddy science and no small amount of bullying has been used in an attempt to ‘silence’ those who dare suggest there is a link between MMR vaccination and autism

    It is your claim that such research is shoddy. Without knowing your criticisms, I cannot refute them. To come to such a conclusion you must have read and found substantial scientific failings in those studies. It should therefore be a relatively simple task to document the individual failings of each study on your blog. So far, your major criticism is a more diffuse criticism that some of the research is epidemiological in nature, which does not deal with the non-epidemiological studies.

    Given the dangers that parents expose their children to by non-vaccination, claims such as these should be substantiated. Yours views are no doubt respected by people making decisions about their children and vaccines.

    There’s also an implication in the above quote that such studies were performed to silence critics. Do you consider this was part of some concerted and organised campaign? Would it have been better if they had not be performed?

    But please do tell us Anthony, how is it that I’m having my cake and eating it?

    I believe this post, and its follow-up post, can be construed as suggesting that doubt exists over the safety of MMR vaccine in relation to autism. It is being taken as such by the members of the JABS forum. Your, arguably technically correct, stance that you are merely pointing out that MMR vaccine has not been not proved to be causing autism is a possible get-out-of-jail-free card. However, for reasons I will outline later it is a weak defense.

    “I note you suggest this evidence ‘undermines’ the hypothesis, but you seem to have stopped short of saying it ‘disproves’ it. Perhaps you can clarify?”

    Wakefield put forward a hypothesis that MMR vaccine caused autism. The Autism-MMR vaccine hypothesis was not supported even by his initial study, and we have since learned how false positives obtained in his study further undermined it. Other studies have been performed in the ten years since he made his extraordinary claim, none of which have confirmed his hypothesis.

    However, no matter how many studies are performed, it is extremely difficult to prove a negative and refute utterly the Autism-MMR vaccine hypothesis. However, the Autism-MMR vaccine hypothesis has taken on the characteristics of Bertrand Russell’s Celestial Teapot.

    The following extract is taken from Susser M. The logic of Sir Karl Popper and the practice of epidemiology. Am J Epidemiol 1986;124:711-718, 1986.

    He [Karl Popper] would allow that if a hypothesis fails successive tests, it is falsified. For him, however, every affirmative result in the same direction indicates no more than survival of the hypothesis. These tests merely expand the range of outcomes that the hypothesis disallows. They do not affirm or verify, nor do they alter the probability that a theory is true.

    In the case of a persisting null result, one can agree and Popper would accept that consistency demands rejection of a hypothesis. Indeed, the inductivists Francis Bacon and John Stuart Mill both argue that the elimination of alternative hypotheses by negative instances contributes more to inference than does the piling up of positive instances. Yet, to “prove” a negative is difficult because alternative qualifying hypotheses are so readily to hand; these render the criterion somewhat less decisive in falsification.

    Given ten years of failures to affirm the Autism-MMR vaccine hypothesis, and plenty of consistency of null results, it seems reasonable to reject the hypothesis.

    Regards

  45. Dr John Briffa 31 May 2008 at 6:41 am #

    Anthony

    “It is your claim that such research is shoddy. Without knowing your criticisms, I cannot refute them. To come to such a conclusion you must have read and found substantial scientific failings in those studies. It should therefore be a relatively simple task to document the individual failings of each study on your blog. So far, your major criticism is a more diffuse criticism that some of the research is epidemiological in nature, which does not deal with the non-epidemiological studies.”

    Do you agree that epidemiological evidence cannot be used to prove (or disprove) causality? I’m assuming with your ‘scientist’ hat on for a moment that you’ll simply answer ‘yes’ to this question (but do correct me if I’m wrong here). So, assuming it’s a ‘yes’, are you prepared to accept that no amount of epidemiological evidence, however voluminous, will really tell us if MMR causes autism or not? So, I reckon those that cite epidemiological evidence as ‘evidence’ that MMR does not cause autism are in fact referring to science that is inadequate, not fit for purpose and therefore ‘shoddy’ (shoddy partly because it’s often scientists that quote this stuff and they really ought to know better).

    Assuming we’re agreed on this, then all we need to do now is look at those non-epidemiological studies you referred to. Now, let’s assume for a moment I know not of what you speak, and what I’ve been referring when I say ‘shoddy science’ is the epidemiological data.

    So, you’re now in a position to show me what an idiot I’ve been all along? Because now you can wheel out all this non-epidemiological stuff you say there is (but to date, have not actually cited).

    “Given the dangers that parents expose their children to by non-vaccination, claims such as these should be substantiated. Yours views are no doubt respected by people making decisions about their children and vaccines.”

    The only claim I’ve made is that we don’t know whether or not MMR can cause autism. So, again, let’s assume I’m quite ignorant, and know nothing of the research. Now, if you refute my claim (and assert that MMR does NOT cause autism) then it is incumbent on YOU to provide the evidence. I don’t mind being wrong on this. Believe me when I tell you I have very publicly shifted my position on certain issues when new evidence came to light and/or to my attention (e.g. saturated fat). You have the opportunity to enlighten me. Please take it. And if you have full text copies of the studies to give me, even better. Because no doubt at some point you will have read them in their entirety.

    But if it turns out that that evidence is not really there, then would it inconvenience you awfully if someone were to go and actually do the relevant work? I mean, if it were to turn out that MMR can cause autism, then all this dragging of heels that seems to have gone on won’t have been helping matters, will it?

    “There’s also an implication in the above quote that such studies were performed to silence critics. Do you consider this was part of some concerted and organised campaign? Would it have been better if they had not be performed?”

    For the record, I think some politicians, scientists and journalists have done their level best to ‘persuade’ people that MMR does not cause autism, when, it is my contention, they can’t have known that. I actually don’t have an opinion on whether it would have been better if previous studies had not been performed, and frankly this is a diversion. I think what matters is not what has been done, but what we do now.

    “I believe this post, and its follow-up post, can be construed as suggesting that doubt exists over the safety of MMR vaccine in relation to autism. It is being taken as such by the members of the JABS forum.”

    Without the appropriate evidence that demonstrates MMR to be safe, I don’t think it’s too much of a stretch to imagine that in some people’s mind (including my own) there is ‘doubt’ about the safety of MMR with respect to autism. Why, does it seem to you that this is in any way a contentious point (it’s blindingly obvious).

    “Your, arguably technically correct, stance that you are merely pointing out that MMR vaccine has not been not proved to be causing autism is a possible get-out-of-jail-free card. However, for reasons I will outline later it is a weak defense.”

    I’m ‘technically correct’ but my ‘defense’ is ‘weak’. You’ve got this the wrong way round again: my claim is we don’t know whether or not MMR can cause autism (that’s where I’m technically correct, right?). So if your position is that I’m wrong on this, it’s up to you to ‘defend’ your position with the appropriate science.

    “Wakefield put forward a hypothesis that MMR vaccine caused autism. The Autism-MMR vaccine hypothesis was not supported even by his initial study, and we have since learned how false positives obtained in his study further undermined it. Other studies have been performed in the ten years since he made his extraordinary claim, none of which have confirmed his hypothesis.”

    So, can I summarise, and say that Wakefield’s theory has NOT been disproven. Because, if it had been, you’d have said so, right?

    “However, no matter how many studies are performed, it is extremely difficult to prove a negative and refute utterly the Autism-MMR vaccine hypothesis. However, the Autism-MMR vaccine hypothesis has taken on the characteristics of Bertrand Russell’s Celestial Teapot.

    The following extract is taken from Susser M. The logic of Sir Karl Popper and the practice of epidemiology. Am J Epidemiol 1986;124:711-718, 1986.

    He [Karl Popper] would allow that if a hypothesis fails successive tests, it is falsified. For him, however, every affirmative result in the same direction indicates no more than survival of the hypothesis. These tests merely expand the range of outcomes that the hypothesis disallows. They do not affirm or verify, nor do they alter the probability that a theory is true.

    In the case of a persisting null result, one can agree and Popper would accept that consistency demands rejection of a hypothesis. Indeed, the inductivists Francis Bacon and John Stuart Mill both argue that the elimination of alternative hypotheses by negative instances contributes more to inference than does the piling up of positive instances. Yet, to “prove” a negative is difficult because alternative qualifying hypotheses are so readily to hand; these render the criterion somewhat less decisive in falsification”

    In the first sentence in the paragraph that mentions Karl Popper, I guess I’d like to see the word ‘appropriate’ slipped in before ‘tests’, would that seem reasonable?

    And, from the second paragraph we learn that it’s difficult to “prove” a negative. Which we knew already, right, so I’m not sure why Francis Bacon and John Stuart Mill’s opinions on all this might possibly are required.

    And their hypothetical opinions, by the way, don’t change these basic facts:

    We have no good evidence that MMR does not cause autism (although I accept you allude to some and now no doubt will provide it).

    And while I agree it’s very hard to ‘prove’ a negative, my impression is, to be honest, that some politicians and scientists, if they were sufficiently motivated, could have tried a lot harder to assure us of MMR’s safety in respect of autism.

    “Given ten years of failures to affirm the Autism-MMR vaccine hypothesis, and plenty of consistency of null results, it seems reasonable to reject the hypothesis.”

    And given 10 years of inadequate science and countless accounts of parents who claim to have witnessed their child regress into an autistic state shortly after receiving the MMR vaccine, it seems reasonable that our scientists and politicians should give this matter the attention it deserves, rather than merely insisting that MMR does not cause autism, and in some instances painting concerned parties as ‘stupid’ or ‘hysterical’ or ‘scientifically illiterate’ or as people just ‘looking for someone or something to blame.’

  46. Anthony 31 May 2008 at 8:05 am #

    Dear John,

    Oh dear. That’s just the sort of response I was expecting.

    I think further meaningful discussion is unlikely.

    Regards

  47. Dr John Briffa 31 May 2008 at 8:32 am #

    Anthony

    Let’s please get to they crux of this:

    1. You, more than once, have alluded to ‘evidence’ that appears to vindicate MMR with regard to any potential to cause autism.

    2. I have, more than once, asked you to provide that ‘evidence’.

    3. It turns out you are unwilling or unable to provide this ‘evidence’

    I asked to see your hand, and it appears it wasn’t such a strong one after all (or perhaps you have no cards at all, we don’t know).

    And now you appear ‘cry off’ from the debate, just when it’s getting really interesting and ‘meaningful’.

    And then you wonder why individuals express doubt about the safety of MMR in relation to autism. Just for the record, it has at least something to do with people like you – who give the distinct impression that MMR does not cause autism, but don’t care to or simply can’t provide the evidence that shows this to be so).

    I can’t wait to see you discuss our exchanges here on your own blog, where I trust you’ll permit me the right of reply.

  48. Anthony 31 May 2008 at 12:55 pm #

    John,

    Are you saying that you don’t even know where to look to find the studies?

    I have the papers on this computer, but I am surprised that you are so sure it is shoddy science when you don’t even know which studies you are talking about. You could start by searching my blog for some of the studies, although not all of them are there.

    I’m not crying off debate, I’m waiting for you to put forward evidence for your claims. which we now know to be based on nothing more than supposition, rather than knowledge of the studies concerned.

    By the way the word evidence does not have scare quotes round it. Do they signify some sort of prejudice at all?

  49. Dr John Briffa 31 May 2008 at 1:37 pm #

    Anthony

    “Are you saying that you don’t even know where to look to find the studies?

    I have the papers on this computer, but I am surprised that you are so sure it is shoddy science when you don’t even know which studies you are talking about. You could start by searching my blog for some of the studies, although not all of them are there.”

    Are you going to answer the questions about the appropriateness of epidemiological evidence in determining cause and effect? Will you accept that it is epidemiological evidence that, in the main, has been used to claim that MMR does not cause autism? And do you accept that from a scientific standpoint, I’m entitled to use the word ‘shoddy’ (as in inadequate) in reference to this research?

    I know where to look for the evidence that is claimed vindicate MMR with regard to autism (or at least I think I do). The thing is, Anthony, it’s you who is putting up these studies, so it’s your job to provide them. I mean, does it really seem reasonable for you to ask me to find the studies you say support your stance? You suggest I search your blog, but might not actually find the studies you’re alluding to there anyway? Do you see how faintly ridiculous this seems?

    Why not just push a few buttons and provide them here?

    As I said, you have a golden opportunity now to show me just how ill-informed I am, by wheeling out all this non-epidemiological evidence you allude to. But yet again you have resisted doing this. Why?

    “I’m not crying off debate, I’m waiting for you to put forward evidence for your claims. which we now know to be based on nothing more than supposition, rather than knowledge of the studies concerned.”

    Please see my comments on epidemiological evidence above and answer the questions I pose. Please. And so what if my opinion is based merely on supposition (which I maintain it is not)? Why not just wade right in (as I’ve invited you to do several times now)? Because that surely would be the way, would it not, to prove that my stance is merely based on supposition. And that surely would be the most expedient way to show us all that MMR does not cause autism, as you appear to claim. So go for it.

    “By the way the word evidence does not have scare quotes round it. Do they signify some sort of prejudice at all?”

    They’re not scare quotes, they’re used here as quotation marks. You used the word, and I’m quoting you. The reason that I’m quoting you is because while you’ve used this word, as yet there isn’t any evidence here that the ‘evidence’ exists.

  50. Anthony 31 May 2008 at 3:19 pm #

    So your claim the science is shoddy is based merely on your observation that epidemiological studies do not prove cause and effect?

    Devastating stuff.

    Your opinion is based on supposition and credulity in the face of emotive anecdotes.

    If it isn’t lay out your detailed critiques of the studies you say do not prove the safety of MMR vaccine.

    The burden of proof lies in your court.

  51. Dr John Briffa 31 May 2008 at 4:23 pm #

    Anthony

    “So your claim the science is shoddy is based merely on your observation that epidemiological studies do not prove cause and effect?
    Devastating stuff.”

    Devastating stuff indeed, when you consider just how often individuals (including scientists – who really ought to know better) have used this epidemiological evidence as though it vindicated MMR with regard to autism. But of course it doesn’t and never will.

    I don’t need to critique the evidence for two main reasons:

    1. That’s already been done, including by the Cochrane reviewers (although even though they we’re less than complimentary about the research some managed to use the review as vindication for their ‘MMR is safe with regard to autism’ stance

    2. It tells us nothing with regard to any putative causal link between MMR and autism

    Would it not therefore be a meaningless intellectual exercise for me to critique the evidence, because no epidemiological evidence will ever be good enough to answer the question we’re asking here.

    Now, let’s imagine for a moment I haven’t looked at any of this research at all (though I can assure you I have), and don’t know anything about it (as you appear to suggest). Can I ask why that would really matter? Sure, now you can claim (as I think you’ve already done, but have provided no proof for ” sounding familiar?) that my assertions are based on supposition. But, even if that were the case:

    1. The claim that I made about the evidence used to vindicate MMR with regard to autism being shoddy still stands (until someone disproves it)

    2. The claim that you made regarding there being good evidence that MMR is safe with respect to autism remains unproven

    In other words, even if I jumped through this hoop, we’d be absolutely none the wiser (and neither would readers here) about whether MMR causes autism or not.

    The only way for one of us to inject any clarity into the situation is for you to either:

    1. Disprove my assertion that the ‘MMR is safe with respect to autism’ line is based on shoddy science

    2. Show us the evidence you say you have but seem reluctant to give us that appears to vindicate MMR with respect to autism.

    Now, then, with regard to my credulity in the face of emotive stories. Well, you see, the thing is about autism is that it can have a devastating effect on children, their families, and everyone around them, So it can get a bit emotional when a parent tells you how they saw their happy, healthy child, shortly after receiving the MMR vaccination, regress into one who cries a lot, stopped talking, won’t be cuddled, and may not even be making eye contact any more. I have to admit, it does genuinely move me. You’re right, maybe I’m a sucker, but my heart genuinely goes out to these people, it really does (no irony intended here).

    Maybe it wouldn’t be so bad if there was just the odd story like this around. But as you know, there are countless numbers of them out there.

    Now, I am not aware of any good evidence that vindicates MMR with regard to autism, so I reckon in the face of considerable anecdotal evidence, that it’s entirely reasonable suggest that our politicians and scientists have been a bit hasty in attempting to persuade the public (not to mention doctors and other health professionals) that MMR does not cause autism.

    Now, I know you’ll disagree, because as you’ve claimed you have the evidence that vindicates MMR with respect to autism. All I ask is that you show it to us.

  52. Anthony 31 May 2008 at 8:30 pm #

    Dear John,

    This is the last comment I am making on your blog, since arguing with wilfully ignorant people is only a minor hobby of mine.

    1. Epidemiology. You don’t like it do you? Funny then that while you do not feel it tells us anything useful about MMR vaccine and autism, you are quite willing to use it when it suits eh?

    Like for example:

    Lof M, et al. Dietary fat and breast cancer risk in fhte [sic] Swedish women’s lifestyle and health cohort. British Journal of Cancer 2007;97:1570-1576

    About which you say:

    Now, it is possible that polyunsaturated and monounsaturated fat do help to reduce breast cancer risk, but a ‘epidemiological’ study of this nature cannot tell us whether this is the case of not. What this study does support is the notion that overall fat intake is unlikely to be an important risk factor for breast cancer (and that women who want to reduce their risk of this condition may not be served by being recommended to cut back on fat).

    So, we both agree that epidemiological studies cannot prove causal effects. You however only believe they are shoddy when applied to vaccines, not when you can use them to suit your views on diet.

    Still, the lack of a correlation between MMR vaccine and autism does give us some indication that the MMR vaccine-autism hypothesis is wrong. Since there are “countless numbers” of cases out there, you would suspect correlations might have arisen in at least one study?

    Indeed, when the alleged “Broad Street Pump handle” of MMR vaccine was removed in Japan autism continued to rise.

    Honda H, Shimizu Y, Rutter M. No effect of MMR withdrawal on the incidence of autism: a total population study. Journal of Child Psychology and Psychiatry 2005;46(6):572-9

    2. Causal links. What would give us a clear causal link between MMR vaccine and autism. Well, a plausible biological mechanism would be nice. There are two virological studies that are of interest. They were looking for the persistence of measles virus which was the proposed mechanism:

    Afzal MA, Ozoemena LC, O’Hare A, Kidger KA, Bentley ML, Minor PD. Absence of detectable measles virus genome sequence in blood of autistic children who have had their MMR vaccination during the routine childhood immunization schedule of UK. Journal of Medical Virology. 2006;78(5):623-30

    D’Souza Y, Fombonne E, Ward BJ, No evidence of persisting Measles virus in peripheral blood mononuclear cells from children with autistic spectrum disorder. Pediatrics 2006;118(4):1164-1675

    The latter paper includes the following information:

    To date, the epidemiologic burden of evidence against such an association in the case of MMR and autism is overwhelming. We now provide evidence that the PCR data published by Uhlmann et al, Martin et al, and Kawashima et al in support of the more limited claim of an association between MMR and a subset of children with ASD (autism spectrum disorder) is also unlikely to be true.

    Our data, together with the epidemiologic evidence, demonstrate that arguments against vaccinating children with MMR because of fear of ASD are not defensible on scientific grounds. The risk of death and disability from MV (measles virus) infection has been unequivocally demonstrated. The hypothesized link between MMR and ASD is spurious and undermines the success of measles control programs.

    It’s even worse than that as well. Because Wakefield’s own PhD student, Chadwick, provided us with the following information under oath:

    Q You state in the affidavit that during your time on your Ph.D. research in Dr. Wakefield’s lab you only obtained nine positive PCR results for measles. Every time you did that you sequenced them?

    A That’s correct, yes. We sent it off to a sequencing lab to be sequenced, and the data that came back showed that they were all false positive results.

    Q Every positive result you got was a false positive?

    A Yes. Yes, apart from the case of the positive control samples which we had, which were a measles infection, a brain disease. We were able to detect measles virus in those cases, so I was confident that the methods were working fine.

    Q Towards the end of your affidavit you state that you had reservations about the immunohistochemistry done to detect measles virus, specifically the use of an antibody from Porton Down?

    A Yes, that’s right. The antibody seemed to cross-react. Experiments we did in the lab seemed to show that the antibody cross-reacted with bacterial proteins, which I think is an artifact of how the antibody was made, and that led us or led me to think that it may have been cross-reacting with bacteria in the gut of patients rather than measles virus.

    Q Now, that would lead to contamination?

    A Well, it would lead to a false positive result. Say for instance if the antibody was binding to something in the guts of these patients, it may well have been a bacteria rather than the measles virus.

    Q Okay. Producing the false positives in those?

    A Yes, that’s correct.

    Q You also state in your affidavit that you believe Dr. Wakefield was aware of all of your negative results when he submitted his paper, “Ileal Lymphonodular Hyperplasia, Nonspecific Colitis and Pervasive Developmental Disorder,” which was published in 1998 to the Lancet.

    A Yes, that’s correct.

    Oh, dear.

    The only shoddy science on display appears to be the science that Wakefield used to start this whole scare off. A scare that you are still concerned about, despite the foundations of it being totally undermined by other science and direct testimony of those involved in it.

    3. Autism. You seem to think your empathy is more important than facts. I have friends with autistic children, and I can assure you that it also moves me.

    However, are these children, and their parents, best served by continuing to propagate a myth, when the reasons for autism are much more complex and requires research? Research, which has probably been delayed by the expensive wild goose chase that has been undertaken over the past ten years, and which has lead to a resurgence of measles in the UK.

    Regards

  53. MinorityView 1 June 2008 at 3:05 pm #

    What I’ve seen in the last 10 years is a very expensive attempt to avoid doing any real research on the autism vaccine connection. For people to now be complaining about the amount of money which has been wasted on bad research which is trying to cover up a connection…is disingenous. Why in the world couldn’t they just look at the children?

  54. Spongebob 1 June 2008 at 4:27 pm #

    How to spend a saturday:

    me – go camping with my ASD child

    PharmaShill – argue, then chicken out when the going gets tough, with Dr John Briffa

  55. Dr John Briffa 1 June 2008 at 5:32 pm #

    Anthony

    You start by insulting me and then claim this is the last comment you’re going to make here. What an opening salvo. I wonder if either of these things is consistent with a person who is confident of their position.

    And then you leap to a conclusion regarding my opinion of epidemiology. For the record, I do think epidemiological evidence can be useful, if associations between things is what you’re trying to assess. But that isn’t the question with MMR and autism, as you very well know: it’s causality that’s the issue here.

    So, if you or someone else (and as you know there’s been many) want to use epidemiological evidence to persuade people that MMR is safe with respect to autism, then my opinion is this is simply not a scientific way to carry on. Because these studies are not fit for purpose. Epidemiological studies cannot be used to disprove the hypothesis that MMR can cause autism. So, in this context, no I don’t ‘like’ epidemiology very much. But that doesn’t mean I don’t like epidemiology: my feelings about it depend on the question we’re trying to answer.

    You go on to quote my assessment of an epidemiological study which assessed the relationship between fat intake and breast cancer. Note my language though, because you’ll see I was circumspect about its findings and their relevance. Because that’s all you can be with a study of this nature. What I didn’t do was make inappropriate inferences and draw inappropriate conclusions from the research, which is what appears to have happened in the case of MMR as it relates to autism.

    And if I may say, you picking out from my site an assessment of some epidemiological study in an effort to demonstrate a purported inconsistency in my approach looks a bit desperate.

    Let’s see if you faired better on the science (because that’s what’s really important here, as the rest is more of a diversion).

    The science

    After repeated asking, you eventually yield the work that I suspect some were anticipating (from the way you were talking) would quash Andrew Wakefield’s original hypothesis once and for all.

    Now, let’s examine in some depth, this long-awaited evidence:

    You cited one epidemiological study – The Honda study

    Are you aware of this critique of this study here?

    No, it’s not in a peer-reviewed journal and maybe you’ll feel its author has no credibility from a scientific perspective. But I’d like you to put these concerns to one side for a moment and not dismiss it out of hand. It’s not the messenger that’s important, but the message, after all. I’m not accusing you of anything here, but good science is partly about being as open and objective as possible, not about coming to snap judgements.

    Please do read this critique in full, as I have. It’s important here to remember that Andrew Wakefield’s original work (however discredited) implicated the measles component of the MMR vaccine in bowel disease (and autism). I know you know this Anthony, but not everyone reading this might.

    If what is stated in it is true, this critique’s most salient points, I think, are:

    In Japan, while MMR vaccination was phased out, measles vaccination continued.

    When both MMR and measles vaccination rates are added together (data from another published study), it appears as though total vaccination rates mirror quite closely the rates of autism and autistic spectrum disorder (ASD) in Japan.

    The relationship between vaccination and autism/ASD appears ‘dose-responsive’ i.e. as vaccination rates go up, so do rates of autism/ASD, and as the rates of vaccination go down, so do rates of autism/ASD.

    I have no way of verifying the data presented here, and even if I could, I think you’d have to describe these findings as ‘preliminary’. But they do look suspicious. I personally would recommend very strongly that this data be subjected to formal study, and published if appropriate.

    With your ‘pharmacoviligance’ hat on now (because as you tell us here your day job is as a ‘pharmacovigilance pharmacist who works at the West Midlands Centre for Adverse Drug Reactions’), imagine someone brought this data to you but it wasn’t about MMR/measles vaccination and autism but, say, heparin injections and migraine. Can I ask you to imagine what your intuitive or even considered response to this data would be?

    I do not draw your attention to this critique because I think, even if the data and their interpretation stand up, that it proves that MMR vaccination causes autism. But you cited the Honda study as evidence of a lack of association between MMR and autism, and I do feel it’s therefore reasonable to suggest, on face of it at least, that the study was hampered by some serious methodological issues. It seems that choice of data was highly biased, and therefore not trustworthy from a scientific perspective.

    The author of the critique makes another interesting point: that one of the authors of this study (Professor Sir Michael Rutter) has acted as a witness for the pharmaceutical industry in litigation cases regarding MMR. Yet, it is alleged, that this was not declared in the paper. If this is correct (I haven’t checked this fact myself at this time), then that would seem to be something of a glaring omission, don’t you think? I mean, there are rules about declaring potential conflicts of interest, and as the author of the critique points out, one of the charges against Andrew Wakefield is that he failed to declare a financial conflict of interest when he published his 1998 study.

    The virological evidence

    You quote two virological studies ” D’Souza and Afzal, both from 2006.

    You’ve demonstrated that you are intimate with the details of Andrew Wakefield’s research, so you will know his original work involved, in layman’s terms (because we need to consider it’s not just you who will be reading this), looking for measles virus in the gut.

    The most striking thing about these studies is that both of them sought to isolate measles in the blood. Does this seem like appropriate science with which to refute Wakefield’s original claims, to you?

    Permit me an analogy: Imagine I claimed I had discovered a new condition characterised by a runny nose and sneezing, and that I’d decided to call this condition ‘the common cold’. I also claim that I’ve been able to isolate a virus ” let’s call it the ‘common cold virus’ – from the noses of people afflicted by this condition.

    Now, if a scientist wanted to attempt to replicate my work, do you think they should attempt to isolate the common cold virus in the nose, or somewhere else, like between the toes or perhaps the anus?

    The answer is glaringly obvious, I think. So, my question to you is, as a scientist, do you not see the fact that these studies looked for measles virus in the blood, rather than a gut, as a fundamental weakness? And do you still maintain that this evidence – how did you put it originally, ‘undermines’ ” Wakefield’s original hypothesis?

    Now, apparently you won’t be posting again here, so we may never get to know your answer. But I can tell you my opinion is that this science is simply inappropriate from a methodological point of view, if proving or disproving Wakefield’s hypothesis is what these scientists had in mind.

    Here’s my summary of the evidence you finally made available:

    1. One epidemiological study that appears to have been very biased in terms of its choice of data selection, and what also looks like a potential conflict of interest regarding one of its authors that went undeclared

    2. Two virological studies that are not fit for the purpose of disproving Andrew Wakefield’s hypothesis

    And remember as I pointed about to you in an earlier comment, even if these studies did ‘disprove’ Andrew Wakefield’s theory, they would not at the same time disprove the theory that MMR can cause autism (because other mechanisms might be at play, right?).

    So, and forgive me if you think I’m being uncharitable (I’m actually attempting to be as objective a possible), the evidence you cite here really does not help to answer the question of whether MMR can cause autism. It most certainly, I believe, cannot be used to assert that MMR does not cause autism.

    After presenting these wholly inadequate studies, you go on to expend quite a lot of column inches on the discrediting of Wakefield’s original work (just saying that this work has been discredited because of false positives would have done, by the way).

    As to the claim that Wakefield knew about this, has anyone verified this? Because if someone has, one feels sure that it would have been gathered up in the bundle of things Andrew Wakefield is being tried for in front of the GMC. To my knowledge, it hasn’t.

    And while we’re on the subject of Andrew Wakefield, are you aware of the following study? It is listed on pubmed and a free full text version is available on-line:

    Uhlmann V, Martin CM, Sheils O, Pilkington L, Silva I, Killalea A, Murch SB, Walker-Smith J, Thomson M, Wakefield AJ, O’Leary JJ. Potential viral pathogenic mechanism for new variant inflammatory bowel disease. Mol Pathol. 2002;55(2):84-90.

    If you’re not familiar with this research already, allow me to summarise: The study looked for the evidence of measles virus in the guts of children diagnosed with developmental disorder and normal children. The researchers discovered evidence of measles virus in the guts of 75 of 91 children with developmental disorder, but in only 5 of 70 children without it (Fisher exact test, p < 0.0001). While this study was in performed in children with developmental disorder (rather than having a diagnosis of autism), I think even the most sceptical person would say that its results are very much in keeping, and in fact if anything strengthen, the findings of Wakefield’s original research. They also raise, again, a question over the safety of MMR. I am also aware of this unpublished study: Walker SJ, Hepner K, Segal J, Krigsman A. Persistent ileal measles virus in a large cohort of regressive autistic children with ileocolitis and lymphnodular hyperplasia: re-visitation of an earlier study [abstract]. International Meeting for Autism Research (IMFAR) 2006. This study was designed to look for the evidence of measles virus in the guts of 275 children. These preliminary findings of 82 children in this sample discovered measles in the guts of 70 of them. As the study is unpublished, I haven’t linked to it. The link to a newspaper report about this study can be found here. I have been unable to ascertain why this study remains unpublished. This might have something to do with the fact that the study remained uncompleted in 2006, and that, as you know, the publication process can be quite drawn-out. While it may not carry as much weight as a published study, it does appear to at least support Wakefield’s original hypothesis. Now, just as I have done with the research you cited, you may want to critique these studies. Apparently, you won’t be doing that here. Perhaps, though, we can continue these discussions on your own site. Or maybe somewhere else. How about a ‘head-to-head’ in the Daily Mail? (That’s not a serious suggestion, by the way. Mind you, if you were game, so would I be). Autism “You seem to think your empathy is more important than facts. I have friends with autistic children, and I can assure you that it also moves me.” Forgive me, but when I read that bit about you having friends with autistic children and how it moves you, it reminded me of when someone is accused of being racist or a homophobe to which they immediately retort: ‘Some of my best friends are black/gay!’. You see, you follow up very quickly with this: “However, are these children, and their parents, best served by continuing to propagate a myth, when the reasons for autism are much more complex and requires research? Research, which has probably been delayed by the expensive wild goose chase that has been undertaken over the past ten years, and which has lead to a resurgence of measles in the UK.” I would say that your use of the word ‘myth’ here is inappropriate. Because we could only describe the MMR-autism link (if one exists) as a myth if it had been disproven beyond all reasonable doubt. Looking at the evidence you’ve presented here and other evidence such as that assessed in the Cochrane review, do you honestly believe that to be the case? Because I don’t, not by a long shot. And against this, we do have endless reports, it seems, of parents who believed their child regressed into an autistic state shortly after receiving the MMR vaccination. Do you not think we owe it to these people, the public at large and future generations to give this matter proper attention? I do, though I accept some will never see it this way. I also feel compelled to point out that the way many people have behaved over this issue has been and continues to be, in my opinion, quite outrageous. None of us are entirely blameless, I suppose, but I am regularly aghast at the hostile and abusive attacks that ‘dissenters’ so often face. The reason that I know this is because I’ve looked in on this area on and off for some time now, and in addition to witnessing what looks to me like a quite brutal battle, I’ve become used to the claims that are made, and the evidence that people used to support them. And when I went to look at that evidence, what I found was something that, in my view, is wholly inadequate, hence why I termed it ‘shoddy’ here. Reading around the subject for some time is, by the way, why I was so confident about calling your hand: I had a pretty good idea what would be in it, and was confident that it would not amount to much. Your branding me as ‘wilfully ignorant’ seems typical of the condescending and abusive tone so often used in this debate. What is it about me that causes you to conclude that I am wilfully ignorant? Is it because when you demanded that I provide the basis for my assertion about ‘shoddy’ science that I did not cough it up on order? Or is your suggestion that my position on MMR and autism is incorrect because it’s not supported by the evidence? Looking at the evidence here and elsewhere, I maintain that it is. And I also maintain that our authorities and scientists have been a bit hasty in their assertion that MMR is safe with regard to autism. Just because I find you challenging and I don’t agree with, Anthony, I do not feel compelled to label you as wilfully ignorant. But it appears to me that you are closed on this topic, and have made your mind up before all the facts are in, as others have done. And it appears there are plenty out there now who share this view, and are not satisfied and will not be pacified and quietened by being belittled, harangued and bullied. But certainly on one level I admire you: I admire you for having the courage to provide your best shots here, and risk them being dismantled. You mention measles rates are rising, which I assume is a reference to MMR uptake receiving a dent. This phenomenon is usually blamed on those who have expressed doubt over the safety of MMR with regard to autism. Is that what you’re attempting to do here? Because if it is, I feel I ought to point out to you that certainly one factor that has contributed to any drop-off in vaccination rates has been the blinkered intransigence of scientists and politicians, who for 10 years have steadfastly refused to give this issue the proper and urgent attention it clearly deserves.

  56. Dr John Briffa 1 June 2008 at 6:12 pm #

    Re comment 53
    I can’t get two of the links to work properly above.

    Link to newspaper report of Walker study is:

    http://www.dailymail.co.uk/news/article-388051/Scientists-fear-MMR-link-autism.html

    Link to the original post where I used the word ‘shoddy’ is:

    http://www.drbriffa.com/blog/2008/05/30/why-the-mmr-autism-war-is-not-over/

  57. Spongebob 1 June 2008 at 6:25 pm #

    Dr John

    Don’t waste your time on Cox – he has a habit of “cutting and running” – even on one of “Pharmacist only” sites (although not this topic). He’s been avoiding Pluralist, Cybertiger and Cliff Miller (and others) like the plague – or should i say measles!

  58. Dr John Briffa 1 June 2008 at 6:31 pm #

    Spongebob
    I fear it is too late, because even after Anthony Cox appeared to cut and run, I felt compelled to answer his last post (geddit?) in excruciating detail – see comment 54.

  59. Spongebob 2 June 2008 at 7:40 am #

    Dr John

    I see that you and Cox had a further exchange on his blog – and as usual he won’t respond to the questions asked.
    But if you persist (on his site) he will block your posts, as happens frequently there. Also happens on other sites such as LeftBrain/No Brain, Goldacres forum, Orac etc. They make sweeping statements, and to prove that these are true they prevent any dissenting voices from posting comments, and hey presto – no dissent = must be true.

  60. Dr John Briffa 2 June 2008 at 8:14 am #

    Spongebob
    Yes, the debate (I call it that for ease, because it’s not really a debate because as you say, my questions are simply going answered) is unfolding here:
    http://www.blacktriangle.org/blog/?p=1799#comment-33804

  61. David 2 June 2008 at 11:54 am #

    Dr Briffa, in an earlier post you dismissed the Peltola papers as not providing any evidence to dispute a causal link between MMR and autism.

    Peltola H, et al. No evidence for measles, mumps, and rubella vaccine-associated inflammatory bowel disease or autism in a 14-year prospective study. Lancet 351(9112);1327-8. This letter was based on data that came from a previous study by the same team: Peltola H, et al. The elimination of indigenous measles, mumps, and rubella from Finland by a 12-year, two-dose vaccination program. N Engl J Med. 1994 331(21):1397-402.

    This study seems to have been used as evidence for a lack of link between MMR and autism because it apparently showed NO cases of autism after millions of MMR administrations. The devil of course, is in the detail: autism cases were NOT MONITORED as part of this. Some other adverse effects were monitored, it seems, but not autism (for some reason). In fact, in the whole of this study, the words ‘autism’ appears precisely NO times.

    The study looked at the consequenses of vaccinating 1.5 million children with MMR, nearly 3 million doses in all. A substudy (a double blind placebo-controlled cross over study in 1162 twins) looked closely at events within 3 weeks of immunisation and found no significant neurological adverse effects. The main study monitored through a surveillance system in an open fashion for any adverse events, recording them whenever they occured.

    The fact that there were no instances of any significant neurological complication is implicit in the studies’ results and cannot be ignored. The authors do not specifically mention “autism” as a named complication, but why mention something if it was not recorded as an event, and only to point out its absence? They did not mention blue tongue disease either – does that mean you feel there is insufficient evidence in this study to conclude blue tongue is not caused by MMR?

    Now I do not dispute the fact that the Peltola studies may not have been methodologically robust enough to ascertain some neurological changes some months after MMR administration, but then that is not what parents claim happen to their children. Their stories often point to immediate, noticeable and significant alterations in behaviour. These are effects that the Peltola surveillance, which was extremely comprehensive, should have picked up over the background incidence. It didn’t.

    We are used to hearing claims that 1 in 100 children develop autism, and that MMR or other vaccines are the cause. In Finland, there should have been up to 15 thousand autism cases precipitated by the vaccine study. The most frequent causally linked side effect of the vaccine was fever in 0.07%, and ITP occured in 0.03%.

    The followup correspondence in the Lancet in 1998 from Peltola’s group looked at those who reported gastrointestinal symptoms after vaccination, and what the long term outcome was for these children. They found no evidence of autistic spectrum disorder in any of these children. The putative mechanism by which MMR is supposed to cause autism is through increased gastrointestinal permeability following inflammation of the gut from measles virus. It seems quite appropriate to conclude that these studies do not show a link between MMR and autism. In science, one can never prove a negative, as you know, but it should be clear to anyone looking at epidemiological studies like Peltola’s that even if MMR does cause autism, it does so in such a tiny proportion that it is indistinguishable from the background noise.

    I am not claiming MMR is “completely safe”, since it clearly is not and it can have adverse effects. However, I doubt precipitating autism is one of them, but I keep an open mind to the possibility that MMR may be a trigger in a tiny percentage of susceptible individuals, in the same way that the natural infection itself (measles) might be a trigger.

    As always, the relevant criterion by which to judge vaccination issues is to look at the risk/benefit equation. For MMR, this is heavily skewed in the favour of vaccination.

  62. Angie 2 June 2008 at 12:00 pm #

    One of the odd things about the whole debate in relation to vaccination is the fact that, for my generation (immediate post-WWII babies), measles, mumps and rubella were regarded as relativel unproblematic, routine childhood diseases. I had the first two as a child, as did most of the other children I knew, spending my time during the measles jumping up and down on the beds with my sister (I had rubella as an adult, which apart from avoiding pregnant friends and acquaintances, was entirely trivial). Overall they weren’t perceived as particularly dangerous diseases (unlike, say, diptheria). Maybe this had something to do with populations who were routinely exposed to these pathogens as against the ‘virgin’ populations now? (I can conceive of several ways in which the immune responses might have been different in that context.) This isn’t to say that no children were damaged by these diseases, but it was very very far from the scare stories we hear now about the consequences of a lack of vaccination.

  63. Dr John Briffa 2 June 2008 at 12:21 pm #

    David

    “I am not claiming MMR is “completely safe”, since it clearly is not and it can have adverse effects. However, I doubt precipitating autism is one of them, but I keep an open mind to the possibility that MMR may be a trigger in a tiny percentage of susceptible individuals, in the same way that the natural infection itself (measles) might be a trigger.”

    How refreshing (honestly and not in an ironic way) to find someone who brings what looks like an open mind to the arena. I agree with your sentiments quoted here, but I’m not sure about the ‘tiny percentage’ bit. I don’t claim to know any more than you. I find the Honda study critique linked to in my post no 54 quite concerning, for instance. I do think scientists should be looking at this to see if the data stacks up.

  64. jdc 2 June 2008 at 3:53 pm #

    There is more discussion of some of Dr Briffa’s points here: http://jdc325.wordpress.com/2008/05/30/more-briffa/

  65. Dr John Briffa 2 June 2008 at 6:27 pm #

    Re: comment 65:

    The author of the apathysketchpad blog, you will discover, a one Andrew Taylor.
    The link that he posts to above goes to a blog post, followed by a discussion he and I had on-line about a number of matters, mainly logic and science. Do, please read the posts, I urge you, because they aptly demonstrate lack of understanding of either of these two disciplines that I’m finding depressingly common in those who purport to be ‘scientists’. See him also demonstrate his very tenuous grasp of the science in the area of MMR and autism. I think, these exchanges demonstrate this particular brand of ignorance so well, that I’m linking to it again should anyone miss it. See here: http://www.apathysketchpad.com/blog/2008/05/30/a-briffas-wrong/

    Now, something you need to know: Andrew Taylor has a dim attitude of parents who believe their child’s autism may have been caused by MMR vaccination. He takes delight, it seems, in belittling such parents. With no proof that their belief is incorrect, he still has no hesitation in pouring scorn on them. How lovely.

    Now, in his defense, Andrew is a scientist, so we should take his word therefore for why parents who believe that MMR vaccination may have caused their child’s autism are deluded and need setting straight through chastisement. You see, expert Andrew is PhD student in 2D and 3D image analysis. How comforting that those who have autistic children to know that they can look to someone so aptly qualified for enlightenment and correction.
    .
    I’m warning you, Andrew Taylor’s site is not for those of a sensitive or caring disposition

  66. jdc 3 June 2008 at 11:16 am #

    “I’m warning you, Andrew Taylor’s site is not for those of a sensitive or caring disposition”
    Dr Briffa likes warning people. His warnings aren’t always so friendly though – he has threatened legal action against several people who commented on my blog.

    Here, John makes the accusations: http://jdc325.wordpress.com/2008/05/30/more-briffa/#comment-1070

    Here, he is provided with evidence of his deletions: http://jdc325.wordpress.com/2008/05/30/more-briffa/#comment-1130

    Will you “hold your hands up” now John?

  67. Dr John Briffa 3 June 2008 at 11:26 am #

    jdc

    I maintain that I have NEVER deleted posts. Now, please permit me some time to check the evidence, and get back to you. It may take a day or two, because I am seeing patients this afternoon and lecturing tomorrow. But I will look into it. Would that be OK?

    In the meantime, you might like to get back to me about those questions I pose to you here, and well as the question posed to you this morning in an email about why you protect your identity.

  68. jdc 3 June 2008 at 12:44 pm #

    I thought I had dealt with the substance of your arguments in my blog post. I do not particularly wish to answer questions about my personal circumstances on either your blog or mine (I think it can often be very boring to do so, I don’t think it can tell you anything useful and frankly I think the discussion of evidence is far more interesting), but I hope I can satisfy your curiosity about my personal situation via email. Black Triangle, Left Brain/Right Brain and Apathy Sketchpad have also answered some of the points you made. For instance, I do not intend to respond on the point you made regarding the Poling case as LB/RB has already done so.

    Perhaps you could highlight here the substantive points in your posts that you feel I have not yet answered?

    Re deleted/unapproved posts – yes, I would be grateful if you would look into this. Perhaps if you explained how you moderate this blog it would be possible to clarify that particular point now?

  69. MinorityReport 4 June 2008 at 3:07 pm #

    Re censored posts, try posting about Goldacre’s conflict of interests. Goldacre writes in the Guardian as if he’s an independent journo, debunking concerns about mobile phones and mmr, and calling people who disagree with him morons. The Guardian hides the fact that he works at the Institute of Psychiatry, where they are funded to produce industry and government propaganda disguised as research.
    http://www.cspinet.org/integrity/watch/200709241.html#3

  70. Sian 6 June 2008 at 5:03 pm #

    I can scarcely believe I have read right to the end of this debate – packed, as it is, with long words which fly right over the top of my blonde head…my question is this:
    Is the ultimate goal of the ever increasing number of baby vaccinations that no child will ever die of a childhood illness? Why are we brainwashed into being terrified of diseases that were a normal part of childhood when I was young.
    Like Angie in the post above, I had measles, mumps and german measles – I got some time off school and lots of jelly. WHAT IS THE PROBLEM? I dont know of anyone who suffered any long term problems from any of the diseases we all caught and gave to each other.
    As a mum of 4 children, I wouldnt like any of them to die or become permanently damaged by measles. Equally I wouldnt like them to fall under the wheels of a bus, be stabbed or beaten to death by a mob or become heroin addicts. But I have to accept the possibility that any of these things could happen.
    Maybe my children will live long enough to suffer a few years of neglect and dementia in a retirement home, perhaps they will die young. Death happens – we might as well get used to it.
    The one thing I DO have proof of: my unvaccinated children are far far healthier than my vaccinated children.

  71. jdc 11 June 2008 at 4:39 pm #

    “I maintain that I have NEVER deleted posts. Now, please permit me some time to check the evidence, and get back to you. It may take a day or two, because I am seeing patients this afternoon and lecturing tomorrow. But I will look into it. Would that be OK?”

    [3rd June 2008 11.26am]

  72. Dr John Briffa 12 June 2008 at 8:08 am #

    jdc

    The claim is I’ve deleted comments. Yet, the ‘evidence’ suggests that comments that were not there at one point are there now.

    In other words, this is not evidence of comment deletion at all.

    It looks to me to be the usual diversion from the true issues. Some dietitians use these tactics a lot, it seems. And they do their profession a massive disservice when they do, I think.

  73. ross 13 June 2008 at 11:24 am #

    Spongebob, comment 59. Do you have any evidence for the comments you make here:

    But if you persist (on his site) he will block your posts, as happens frequently there. Also happens on other sites such as LeftBrain/No Brain, Goldacres forum, Orac etc. They make sweeping statements, and to prove that these are true they prevent any dissenting voices from posting comments, and hey presto – no dissent = must be true.

    Dr B – so what was the reason for the magic reappearance of the missing comments? Were they deleted and re-instated or was there a very long comment moderation period?

  74. Richard 24 June 2008 at 3:56 pm #

    Some sense at last from Sian post No:71. I too have read this post to the end (phew!) and like the road accident discussed, it has been hard to not look and see the impending mess.
    I have three unvaccinated boys 3, 9 & 11 years old. We have only had one trip to the doctor out of all those years, never used antibiotics, do not use fluoride in toothpastes and none of them have fillings. This is replicated by all other non-vaccination families I know and am friends with. They are the healthiest kids I know and illness is seen as something important to the developing immune system (mumps was no bother). Other then non vaccination, no antibiotics (oral and household sprays), no fluoride and reduced chemicals (esp artificial sweeteners) and toxins in food and house, I can see no other things that we do differently to other friends. Why are the majority of my neighbours’ kids regularly very ill, on continuous rounds of antibiotics, steroids, suffering from asthma, autism, allergies. It may not all lie at the feet of vaccination but it is just another assault on the immune system.

    One question though – Siam maybe you have an idea? Now that these diseases are harder to find, my kids have not had measles and I am not sure if I want to risk them having it during teenage due to the increased dangers of getting the disease naturally but later than normal, so should I have the oldest one vaccinated by single dose with Measles (they have had mumps naturally – very easily). Can’t decide what to do? It is healthier to get the disease naturally but may be risky to get it as a teenager or adult?

Trackbacks/Pingbacks

  1. Apathy Sketchpad » Blog Archive » A Briffa’s Wrong - 2 June 2008

    [...] other day I posted about Dr John Briffa’s rant against p-values. He has since then posted some responses, in the form of several comments under the original post [...]

  2. AltMed Support for Wakefield Continues « jdc325’s Weblog - 15 March 2009

    [...] Dr John Briffa has also indulged in some MMR scaremongering of his own (Briffa’s original post), which I covered here, here, and here [note: the first of my blog posts includes links to many [...]

  3. Adverse effects of drugs are “neglected, restricted, distorted and silenced” | Hard Sweat - 17 February 2010

    [...] Dr Ioannidis also makes the point some trials limit their reporting of adverse effects to ‘statistically significant’ results. The problem is, the most serious adverse effects are generally very rare, and studies may not be large (powered) enough to define serious adverse effects as ‘statistically significant’. Dr Ioannidis claims that the statements such as “no significant differences were found” are misleading. I made a similar point some time ago in the blog post here. [...]

Leave a Reply