The limited value of ‘statistical significance’ in the real World

Earlier this week I was working from home in the morning. I had the radio on in the background. My normal choice of oral wallpaper is BBC Radio 4. It’s often on, but I’m rarely ‘listening’ to it. I rely on the ‘cocktail party effect’ to pick up on anything vaguely relevant. In other words, listening to the radio for me is a bit like being at a gathering where there is a buzz of conversation in the background. In the main, what we ‘hear’ is filtered out, but if someone where to say something particularly relevant to us, such as our name, then it would tend to attract our attention. So, I am generally deaf to items on such matters as the spotting of a rare bird on a remote Scottish island and the minutiae of fiscal policy here in the UK, but when something about health or science pops up, I can suddenly be all ears.

So, earlier this week my attention was grabbed by an item on the debate about whether the time limit for termination of pregnancy should be dropped from 24 weeks (where it stands now) to 22 or 20 weeks. By the way, this blog article is not ethically, morally or religiously driven, it about science, or rather, the limitations of it.

One side of the argument here states that the abortion time limit should be brought down because babies can (and do) survive when born at an age lower than the current 24-week cut-off. Those opposing the change have generally used the argument that the ‘evidence’ shows that the survivability of infants born very prematurely has not changed in recent years. So, if 24 weeks was good enough when the limit was set, it is good enough now.

The obvious riposte to Gordon Brown’s (and others’) ‘scientifically-based’ argument is that there’s no reason to assume that just because the survivability of very premature infants has not changed, that the abortion time limit right. Maybe we got it ‘wrong’ the first time round and there’s an argument for reviewing the limit.

One individual supporting a review was a woman who was interviewed on Radio 4 who, if I remember correctly, delivered a child at 22 weeks gestation. The child was, she said, left to die. However, because after 36 hours this child had not died, it was duly treated with medical care and survived. According to the mother, while the child was (naturally) a slow-starter, he had caught up and was leading the sort of life you’d expect ‘normal’ children to lead.

It was put to her by the interviewer that infant mortality statistics had not changed, so how could she justify her desire (as if it were not obvious) for the termination time limit to be reduced. What she said, and I’m doing this from memory, was, I think, very telling. She first of all suggested that we need to be a bit careful with statistics. She reiterated the point that children can survive at an age lower than the termination limit. She rounded this off by suggesting that while the statistics may not have changed significantly, for a child who may survive being born very prematurely the issue is very significant indeed.

I think she has a point. And this whole issue reminds me of just how easily we over-rely on the science and statistics. And examples of this, I think, are legion in the medical field.

For example, I have written before about the placebo response and its power in promoting healing. Some (for instance, academics who never go near real patients) dismiss the placebo response as an artefact, and something that is not ‘real’ like the effect you get with, say, a drug that has been ‘proven’ to be effective. My opinion is that if a treatment or approach helps someone, the mechanism behind the improvement is far less important than the fact that they have improved. But I suppose that’s one of the differences between academics and individuals who actually see patients with real problems and who are focused on actually helping people.

Another way science may not be of service of us concerns ‘statistical significance’. This tells us, supposedly, whether there’s some real effect or change going on, or it’s merely something that’s most likely to be due to chance. Statistical significance in scientific studies is denoted by what is known as the P (or probability) value. A value of less than 0.05 is generally regarded as denoting ‘statistical significance’.

Sounds fine so far. Except, I do feel compelled to point out that the choice of 0.05 as a cut-off is utterly arbitrary. It’s a value that the scientific community agree on. It’s a consensus ” it’s not carved in stone like some irrefutable scientific truth. If the scientific community decided that 0.01 was going to be a cut-off, then less things would be ‘statistically significant’. If the limit was set at 0.1 then many more things would be deemed significant. When we understand this, we begin to see just how arbitrary a lot of scientific ‘findings’ really are.

An example of where statistical significance appears to have got in the way of a constructive debate on the subject is vaccination. Our Government here in the UK, most doctors (I suspect) and many commentators would have us believe that vaccination, including the measles, mumps and rubella vaccination (MMR) is ‘safe’. Many will not even entertain the thought that there may be a problem with MMR. They’ll quote the science (some of which is not of the highest quality anyway) in a way that gives the impression, very often, that there is NOTHING AT ALL to worry about.

An analogy may be useful here. Let’s imagine someone decided to do a big study on road safety. Let’s say they counted up the number of times someone, somewhere, crossed the road. And now, let’s imagine, they also count up the number of times someone gets run over (and hurt or killed) as a result of crossing the road. Now, I’m writing this on a plane and can’t even check if these statistics exist. But I think it’s reasonable to assume, that compared to the total number of road crossings, the number of people being knocked down is likely to be very small indeed.

Now imagine we applied some statistical ‘wizardry’ to this (with that arbitrary P value, remember) It’s not too difficult to imagine that one would turn up a result which shows: ‘crossing the road is not associated with a statistically significant increased risk of getting run over.’ Now, many doctors and scientists would interpret this finding as evidence that crossing the road is ‘safe’. However, we all know that while most of the time it is, sometimes it’s not.

Now, getting run over has obvious after effects. Vaccination, on the other hand, may not. The effect, for instance, may be delayed. And also the changes can be more subtle than a broken leg, a ruptured spleen or death. Nevertheless, despite the protestations of some, there is a considerable body of people out there who believe (rightly or wrongly) that their child has been damaged by vaccination. And all too often these individuals are dismissed or patronised.

To get some indication of how some of these parents might feel, imagine for a moment turning up at hospital with your child who has been run over. When you get to casualty the attending doctor asks what happened to your child. You reply that they were run over crossing the road. Now imagine the doctor turns round to you and says with a somewhat withering tone: I don’t think so: Study after study shows that there’s no credible evidence that crossing the road can be harmful to human health.�

Whatever scientists and doctors sometimes contend, the fact remains: accidents can happen.

77 Responses to The limited value of ‘statistical significance’ in the real World

  1. Dr John Briffa 31 May 2008 at 4:23 pm #

    Anthony

    “So your claim the science is shoddy is based merely on your observation that epidemiological studies do not prove cause and effect?
    Devastating stuff.”

    Devastating stuff indeed, when you consider just how often individuals (including scientists – who really ought to know better) have used this epidemiological evidence as though it vindicated MMR with regard to autism. But of course it doesn’t and never will.

    I don’t need to critique the evidence for two main reasons:

    1. That’s already been done, including by the Cochrane reviewers (although even though they we’re less than complimentary about the research some managed to use the review as vindication for their ‘MMR is safe with regard to autism’ stance

    2. It tells us nothing with regard to any putative causal link between MMR and autism

    Would it not therefore be a meaningless intellectual exercise for me to critique the evidence, because no epidemiological evidence will ever be good enough to answer the question we’re asking here.

    Now, let’s imagine for a moment I haven’t looked at any of this research at all (though I can assure you I have), and don’t know anything about it (as you appear to suggest). Can I ask why that would really matter? Sure, now you can claim (as I think you’ve already done, but have provided no proof for ” sounding familiar?) that my assertions are based on supposition. But, even if that were the case:

    1. The claim that I made about the evidence used to vindicate MMR with regard to autism being shoddy still stands (until someone disproves it)

    2. The claim that you made regarding there being good evidence that MMR is safe with respect to autism remains unproven

    In other words, even if I jumped through this hoop, we’d be absolutely none the wiser (and neither would readers here) about whether MMR causes autism or not.

    The only way for one of us to inject any clarity into the situation is for you to either:

    1. Disprove my assertion that the ‘MMR is safe with respect to autism’ line is based on shoddy science

    2. Show us the evidence you say you have but seem reluctant to give us that appears to vindicate MMR with respect to autism.

    Now, then, with regard to my credulity in the face of emotive stories. Well, you see, the thing is about autism is that it can have a devastating effect on children, their families, and everyone around them, So it can get a bit emotional when a parent tells you how they saw their happy, healthy child, shortly after receiving the MMR vaccination, regress into one who cries a lot, stopped talking, won’t be cuddled, and may not even be making eye contact any more. I have to admit, it does genuinely move me. You’re right, maybe I’m a sucker, but my heart genuinely goes out to these people, it really does (no irony intended here).

    Maybe it wouldn’t be so bad if there was just the odd story like this around. But as you know, there are countless numbers of them out there.

    Now, I am not aware of any good evidence that vindicates MMR with regard to autism, so I reckon in the face of considerable anecdotal evidence, that it’s entirely reasonable suggest that our politicians and scientists have been a bit hasty in attempting to persuade the public (not to mention doctors and other health professionals) that MMR does not cause autism.

    Now, I know you’ll disagree, because as you’ve claimed you have the evidence that vindicates MMR with respect to autism. All I ask is that you show it to us.

  2. Anthony 31 May 2008 at 8:30 pm #

    Dear John,

    This is the last comment I am making on your blog, since arguing with wilfully ignorant people is only a minor hobby of mine.

    1. Epidemiology. You don’t like it do you? Funny then that while you do not feel it tells us anything useful about MMR vaccine and autism, you are quite willing to use it when it suits eh?

    Like for example:

    Lof M, et al. Dietary fat and breast cancer risk in fhte [sic] Swedish women’s lifestyle and health cohort. British Journal of Cancer 2007;97:1570-1576

    About which you say:

    Now, it is possible that polyunsaturated and monounsaturated fat do help to reduce breast cancer risk, but a ‘epidemiological’ study of this nature cannot tell us whether this is the case of not. What this study does support is the notion that overall fat intake is unlikely to be an important risk factor for breast cancer (and that women who want to reduce their risk of this condition may not be served by being recommended to cut back on fat).

    So, we both agree that epidemiological studies cannot prove causal effects. You however only believe they are shoddy when applied to vaccines, not when you can use them to suit your views on diet.

    Still, the lack of a correlation between MMR vaccine and autism does give us some indication that the MMR vaccine-autism hypothesis is wrong. Since there are “countless numbers” of cases out there, you would suspect correlations might have arisen in at least one study?

    Indeed, when the alleged “Broad Street Pump handle” of MMR vaccine was removed in Japan autism continued to rise.

    Honda H, Shimizu Y, Rutter M. No effect of MMR withdrawal on the incidence of autism: a total population study. Journal of Child Psychology and Psychiatry 2005;46(6):572-9

    2. Causal links. What would give us a clear causal link between MMR vaccine and autism. Well, a plausible biological mechanism would be nice. There are two virological studies that are of interest. They were looking for the persistence of measles virus which was the proposed mechanism:

    Afzal MA, Ozoemena LC, O’Hare A, Kidger KA, Bentley ML, Minor PD. Absence of detectable measles virus genome sequence in blood of autistic children who have had their MMR vaccination during the routine childhood immunization schedule of UK. Journal of Medical Virology. 2006;78(5):623-30

    D’Souza Y, Fombonne E, Ward BJ, No evidence of persisting Measles virus in peripheral blood mononuclear cells from children with autistic spectrum disorder. Pediatrics 2006;118(4):1164-1675

    The latter paper includes the following information:

    To date, the epidemiologic burden of evidence against such an association in the case of MMR and autism is overwhelming. We now provide evidence that the PCR data published by Uhlmann et al, Martin et al, and Kawashima et al in support of the more limited claim of an association between MMR and a subset of children with ASD (autism spectrum disorder) is also unlikely to be true.

    Our data, together with the epidemiologic evidence, demonstrate that arguments against vaccinating children with MMR because of fear of ASD are not defensible on scientific grounds. The risk of death and disability from MV (measles virus) infection has been unequivocally demonstrated. The hypothesized link between MMR and ASD is spurious and undermines the success of measles control programs.

    It’s even worse than that as well. Because Wakefield’s own PhD student, Chadwick, provided us with the following information under oath:

    Q You state in the affidavit that during your time on your Ph.D. research in Dr. Wakefield’s lab you only obtained nine positive PCR results for measles. Every time you did that you sequenced them?

    A That’s correct, yes. We sent it off to a sequencing lab to be sequenced, and the data that came back showed that they were all false positive results.

    Q Every positive result you got was a false positive?

    A Yes. Yes, apart from the case of the positive control samples which we had, which were a measles infection, a brain disease. We were able to detect measles virus in those cases, so I was confident that the methods were working fine.

    Q Towards the end of your affidavit you state that you had reservations about the immunohistochemistry done to detect measles virus, specifically the use of an antibody from Porton Down?

    A Yes, that’s right. The antibody seemed to cross-react. Experiments we did in the lab seemed to show that the antibody cross-reacted with bacterial proteins, which I think is an artifact of how the antibody was made, and that led us or led me to think that it may have been cross-reacting with bacteria in the gut of patients rather than measles virus.

    Q Now, that would lead to contamination?

    A Well, it would lead to a false positive result. Say for instance if the antibody was binding to something in the guts of these patients, it may well have been a bacteria rather than the measles virus.

    Q Okay. Producing the false positives in those?

    A Yes, that’s correct.

    Q You also state in your affidavit that you believe Dr. Wakefield was aware of all of your negative results when he submitted his paper, “Ileal Lymphonodular Hyperplasia, Nonspecific Colitis and Pervasive Developmental Disorder,” which was published in 1998 to the Lancet.

    A Yes, that’s correct.

    Oh, dear.

    The only shoddy science on display appears to be the science that Wakefield used to start this whole scare off. A scare that you are still concerned about, despite the foundations of it being totally undermined by other science and direct testimony of those involved in it.

    3. Autism. You seem to think your empathy is more important than facts. I have friends with autistic children, and I can assure you that it also moves me.

    However, are these children, and their parents, best served by continuing to propagate a myth, when the reasons for autism are much more complex and requires research? Research, which has probably been delayed by the expensive wild goose chase that has been undertaken over the past ten years, and which has lead to a resurgence of measles in the UK.

    Regards

  3. MinorityView 1 June 2008 at 3:05 pm #

    What I’ve seen in the last 10 years is a very expensive attempt to avoid doing any real research on the autism vaccine connection. For people to now be complaining about the amount of money which has been wasted on bad research which is trying to cover up a connection…is disingenous. Why in the world couldn’t they just look at the children?

  4. Spongebob 1 June 2008 at 4:27 pm #

    How to spend a saturday:

    me – go camping with my ASD child

    PharmaShill – argue, then chicken out when the going gets tough, with Dr John Briffa

  5. Dr John Briffa 1 June 2008 at 5:32 pm #

    Anthony

    You start by insulting me and then claim this is the last comment you’re going to make here. What an opening salvo. I wonder if either of these things is consistent with a person who is confident of their position.

    And then you leap to a conclusion regarding my opinion of epidemiology. For the record, I do think epidemiological evidence can be useful, if associations between things is what you’re trying to assess. But that isn’t the question with MMR and autism, as you very well know: it’s causality that’s the issue here.

    So, if you or someone else (and as you know there’s been many) want to use epidemiological evidence to persuade people that MMR is safe with respect to autism, then my opinion is this is simply not a scientific way to carry on. Because these studies are not fit for purpose. Epidemiological studies cannot be used to disprove the hypothesis that MMR can cause autism. So, in this context, no I don’t ‘like’ epidemiology very much. But that doesn’t mean I don’t like epidemiology: my feelings about it depend on the question we’re trying to answer.

    You go on to quote my assessment of an epidemiological study which assessed the relationship between fat intake and breast cancer. Note my language though, because you’ll see I was circumspect about its findings and their relevance. Because that’s all you can be with a study of this nature. What I didn’t do was make inappropriate inferences and draw inappropriate conclusions from the research, which is what appears to have happened in the case of MMR as it relates to autism.

    And if I may say, you picking out from my site an assessment of some epidemiological study in an effort to demonstrate a purported inconsistency in my approach looks a bit desperate.

    Let’s see if you faired better on the science (because that’s what’s really important here, as the rest is more of a diversion).

    The science

    After repeated asking, you eventually yield the work that I suspect some were anticipating (from the way you were talking) would quash Andrew Wakefield’s original hypothesis once and for all.

    Now, let’s examine in some depth, this long-awaited evidence:

    You cited one epidemiological study – The Honda study

    Are you aware of this critique of this study here?

    No, it’s not in a peer-reviewed journal and maybe you’ll feel its author has no credibility from a scientific perspective. But I’d like you to put these concerns to one side for a moment and not dismiss it out of hand. It’s not the messenger that’s important, but the message, after all. I’m not accusing you of anything here, but good science is partly about being as open and objective as possible, not about coming to snap judgements.

    Please do read this critique in full, as I have. It’s important here to remember that Andrew Wakefield’s original work (however discredited) implicated the measles component of the MMR vaccine in bowel disease (and autism). I know you know this Anthony, but not everyone reading this might.

    If what is stated in it is true, this critique’s most salient points, I think, are:

    In Japan, while MMR vaccination was phased out, measles vaccination continued.

    When both MMR and measles vaccination rates are added together (data from another published study), it appears as though total vaccination rates mirror quite closely the rates of autism and autistic spectrum disorder (ASD) in Japan.

    The relationship between vaccination and autism/ASD appears ‘dose-responsive’ i.e. as vaccination rates go up, so do rates of autism/ASD, and as the rates of vaccination go down, so do rates of autism/ASD.

    I have no way of verifying the data presented here, and even if I could, I think you’d have to describe these findings as ‘preliminary’. But they do look suspicious. I personally would recommend very strongly that this data be subjected to formal study, and published if appropriate.

    With your ‘pharmacoviligance’ hat on now (because as you tell us here your day job is as a ‘pharmacovigilance pharmacist who works at the West Midlands Centre for Adverse Drug Reactions’), imagine someone brought this data to you but it wasn’t about MMR/measles vaccination and autism but, say, heparin injections and migraine. Can I ask you to imagine what your intuitive or even considered response to this data would be?

    I do not draw your attention to this critique because I think, even if the data and their interpretation stand up, that it proves that MMR vaccination causes autism. But you cited the Honda study as evidence of a lack of association between MMR and autism, and I do feel it’s therefore reasonable to suggest, on face of it at least, that the study was hampered by some serious methodological issues. It seems that choice of data was highly biased, and therefore not trustworthy from a scientific perspective.

    The author of the critique makes another interesting point: that one of the authors of this study (Professor Sir Michael Rutter) has acted as a witness for the pharmaceutical industry in litigation cases regarding MMR. Yet, it is alleged, that this was not declared in the paper. If this is correct (I haven’t checked this fact myself at this time), then that would seem to be something of a glaring omission, don’t you think? I mean, there are rules about declaring potential conflicts of interest, and as the author of the critique points out, one of the charges against Andrew Wakefield is that he failed to declare a financial conflict of interest when he published his 1998 study.

    The virological evidence

    You quote two virological studies ” D’Souza and Afzal, both from 2006.

    You’ve demonstrated that you are intimate with the details of Andrew Wakefield’s research, so you will know his original work involved, in layman’s terms (because we need to consider it’s not just you who will be reading this), looking for measles virus in the gut.

    The most striking thing about these studies is that both of them sought to isolate measles in the blood. Does this seem like appropriate science with which to refute Wakefield’s original claims, to you?

    Permit me an analogy: Imagine I claimed I had discovered a new condition characterised by a runny nose and sneezing, and that I’d decided to call this condition ‘the common cold’. I also claim that I’ve been able to isolate a virus ” let’s call it the ‘common cold virus’ – from the noses of people afflicted by this condition.

    Now, if a scientist wanted to attempt to replicate my work, do you think they should attempt to isolate the common cold virus in the nose, or somewhere else, like between the toes or perhaps the anus?

    The answer is glaringly obvious, I think. So, my question to you is, as a scientist, do you not see the fact that these studies looked for measles virus in the blood, rather than a gut, as a fundamental weakness? And do you still maintain that this evidence – how did you put it originally, ‘undermines’ ” Wakefield’s original hypothesis?

    Now, apparently you won’t be posting again here, so we may never get to know your answer. But I can tell you my opinion is that this science is simply inappropriate from a methodological point of view, if proving or disproving Wakefield’s hypothesis is what these scientists had in mind.

    Here’s my summary of the evidence you finally made available:

    1. One epidemiological study that appears to have been very biased in terms of its choice of data selection, and what also looks like a potential conflict of interest regarding one of its authors that went undeclared

    2. Two virological studies that are not fit for the purpose of disproving Andrew Wakefield’s hypothesis

    And remember as I pointed about to you in an earlier comment, even if these studies did ‘disprove’ Andrew Wakefield’s theory, they would not at the same time disprove the theory that MMR can cause autism (because other mechanisms might be at play, right?).

    So, and forgive me if you think I’m being uncharitable (I’m actually attempting to be as objective a possible), the evidence you cite here really does not help to answer the question of whether MMR can cause autism. It most certainly, I believe, cannot be used to assert that MMR does not cause autism.

    After presenting these wholly inadequate studies, you go on to expend quite a lot of column inches on the discrediting of Wakefield’s original work (just saying that this work has been discredited because of false positives would have done, by the way).

    As to the claim that Wakefield knew about this, has anyone verified this? Because if someone has, one feels sure that it would have been gathered up in the bundle of things Andrew Wakefield is being tried for in front of the GMC. To my knowledge, it hasn’t.

    And while we’re on the subject of Andrew Wakefield, are you aware of the following study? It is listed on pubmed and a free full text version is available on-line:

    Uhlmann V, Martin CM, Sheils O, Pilkington L, Silva I, Killalea A, Murch SB, Walker-Smith J, Thomson M, Wakefield AJ, O’Leary JJ. Potential viral pathogenic mechanism for new variant inflammatory bowel disease. Mol Pathol. 2002;55(2):84-90.

    If you’re not familiar with this research already, allow me to summarise: The study looked for the evidence of measles virus in the guts of children diagnosed with developmental disorder and normal children. The researchers discovered evidence of measles virus in the guts of 75 of 91 children with developmental disorder, but in only 5 of 70 children without it (Fisher exact test, p < 0.0001).

    While this study was in performed in children with developmental disorder (rather than having a diagnosis of autism), I think even the most sceptical person would say that its results are very much in keeping, and in fact if anything strengthen, the findings of Wakefield’s original research. They also raise, again, a question over the safety of MMR.

    I am also aware of this unpublished study:

    Walker SJ, Hepner K, Segal J, Krigsman A. Persistent ileal measles virus in a large cohort of regressive autistic children with ileocolitis and lymphnodular hyperplasia: re-visitation of an earlier study [abstract]. International Meeting for Autism Research (IMFAR) 2006.

    This study was designed to look for the evidence of measles virus in the guts of 275 children. These preliminary findings of 82 children in this sample discovered measles in the guts of 70 of them. As the study is unpublished, I haven’t linked to it. The link to a newspaper report about this study can be found here.

    I have been unable to ascertain why this study remains unpublished. This might have something to do with the fact that the study remained uncompleted in 2006, and that, as you know, the publication process can be quite drawn-out. While it may not carry as much weight as a published study, it does appear to at least support Wakefield’s original hypothesis.

    Now, just as I have done with the research you cited, you may want to critique these studies. Apparently, you won’t be doing that here. Perhaps, though, we can continue these discussions on your own site. Or maybe somewhere else. How about a ‘head-to-head’ in the Daily Mail? (That’s not a serious suggestion, by the way. Mind you, if you were game, so would I be).

    Autism

    “You seem to think your empathy is more important than facts. I have friends with autistic children, and I can assure you that it also moves me.”

    Forgive me, but when I read that bit about you having friends with autistic children and how it moves you, it reminded me of when someone is accused of being racist or a homophobe to which they immediately retort: ‘Some of my best friends are black/gay!’. You see, you follow up very quickly with this:

    “However, are these children, and their parents, best served by continuing to propagate a myth, when the reasons for autism are much more complex and requires research? Research, which has probably been delayed by the expensive wild goose chase that has been undertaken over the past ten years, and which has lead to a resurgence of measles in the UK.”

    I would say that your use of the word ‘myth’ here is inappropriate. Because we could only describe the MMR-autism link (if one exists) as a myth if it had been disproven beyond all reasonable doubt. Looking at the evidence you’ve presented here and other evidence such as that assessed in the Cochrane review, do you honestly believe that to be the case? Because I don’t, not by a long shot.

    And against this, we do have endless reports, it seems, of parents who believed their child regressed into an autistic state shortly after receiving the MMR vaccination. Do you not think we owe it to these people, the public at large and future generations to give this matter proper attention? I do, though I accept some will never see it this way.

    I also feel compelled to point out that the way many people have behaved over this issue has been and continues to be, in my opinion, quite outrageous. None of us are entirely blameless, I suppose, but I am regularly aghast at the hostile and abusive attacks that ‘dissenters’ so often face.

    The reason that I know this is because I’ve looked in on this area on and off for some time now, and in addition to witnessing what looks to me like a quite brutal battle, I’ve become used to the claims that are made, and the evidence that people used to support them. And when I went to look at that evidence, what I found was something that, in my view, is wholly inadequate, hence why I termed it ‘shoddy’ here.

    Reading around the subject for some time is, by the way, why I was so confident about calling your hand: I had a pretty good idea what would be in it, and was confident that it would not amount to much.

    Your branding me as ‘wilfully ignorant’ seems typical of the condescending and abusive tone so often used in this debate. What is it about me that causes you to conclude that I am wilfully ignorant? Is it because when you demanded that I provide the basis for my assertion about ‘shoddy’ science that I did not cough it up on order? Or is your suggestion that my position on MMR and autism is incorrect because it’s not supported by the evidence? Looking at the evidence here and elsewhere, I maintain that it is. And I also maintain that our authorities and scientists have been a bit hasty in their assertion that MMR is safe with regard to autism.

    Just because I find you challenging and I don’t agree with, Anthony, I do not feel compelled to label you as wilfully ignorant. But it appears to me that you are closed on this topic, and have made your mind up before all the facts are in, as others have done. And it appears there are plenty out there now who share this view, and are not satisfied and will not be pacified and quietened by being belittled, harangued and bullied.

    But certainly on one level I admire you: I admire you for having the courage to provide your best shots here, and risk them being dismantled.

    You mention measles rates are rising, which I assume is a reference to MMR uptake receiving a dent. This phenomenon is usually blamed on those who have expressed doubt over the safety of MMR with regard to autism. Is that what you’re attempting to do here? Because if it is, I feel I ought to point out to you that certainly one factor that has contributed to any drop-off in vaccination rates has been the blinkered intransigence of scientists and politicians, who for 10 years have steadfastly refused to give this issue the proper and urgent attention it clearly deserves.

  6. Dr John Briffa 1 June 2008 at 6:12 pm #

    Re comment 53
    I can’t get two of the links to work properly above.

    Link to newspaper report of Walker study is:

    http://www.dailymail.co.uk/news/article-388051/Scientists-fear-MMR-link-autism.html

    Link to the original post where I used the word ‘shoddy’ is:

    http://www.drbriffa.com/blog/2008/05/30/why-the-mmr-autism-war-is-not-over/

  7. Spongebob 1 June 2008 at 6:25 pm #

    Dr John

    Don’t waste your time on Cox – he has a habit of “cutting and running” – even on one of “Pharmacist only” sites (although not this topic). He’s been avoiding Pluralist, Cybertiger and Cliff Miller (and others) like the plague – or should i say measles!

  8. Dr John Briffa 1 June 2008 at 6:31 pm #

    Spongebob
    I fear it is too late, because even after Anthony Cox appeared to cut and run, I felt compelled to answer his last post (geddit?) in excruciating detail – see comment 54.

  9. Spongebob 2 June 2008 at 7:40 am #

    Dr John

    I see that you and Cox had a further exchange on his blog – and as usual he won’t respond to the questions asked.
    But if you persist (on his site) he will block your posts, as happens frequently there. Also happens on other sites such as LeftBrain/No Brain, Goldacres forum, Orac etc. They make sweeping statements, and to prove that these are true they prevent any dissenting voices from posting comments, and hey presto – no dissent = must be true.

  10. Dr John Briffa 2 June 2008 at 8:14 am #

    Spongebob
    Yes, the debate (I call it that for ease, because it’s not really a debate because as you say, my questions are simply going answered) is unfolding here:
    http://www.blacktriangle.org/blog/?p=1799#comment-33804

  11. David 2 June 2008 at 11:54 am #

    Dr Briffa, in an earlier post you dismissed the Peltola papers as not providing any evidence to dispute a causal link between MMR and autism.

    Peltola H, et al. No evidence for measles, mumps, and rubella vaccine-associated inflammatory bowel disease or autism in a 14-year prospective study. Lancet 351(9112);1327-8. This letter was based on data that came from a previous study by the same team: Peltola H, et al. The elimination of indigenous measles, mumps, and rubella from Finland by a 12-year, two-dose vaccination program. N Engl J Med. 1994 331(21):1397-402.

    This study seems to have been used as evidence for a lack of link between MMR and autism because it apparently showed NO cases of autism after millions of MMR administrations. The devil of course, is in the detail: autism cases were NOT MONITORED as part of this. Some other adverse effects were monitored, it seems, but not autism (for some reason). In fact, in the whole of this study, the words ‘autism’ appears precisely NO times.

    The study looked at the consequenses of vaccinating 1.5 million children with MMR, nearly 3 million doses in all. A substudy (a double blind placebo-controlled cross over study in 1162 twins) looked closely at events within 3 weeks of immunisation and found no significant neurological adverse effects. The main study monitored through a surveillance system in an open fashion for any adverse events, recording them whenever they occured.

    The fact that there were no instances of any significant neurological complication is implicit in the studies’ results and cannot be ignored. The authors do not specifically mention “autism” as a named complication, but why mention something if it was not recorded as an event, and only to point out its absence? They did not mention blue tongue disease either – does that mean you feel there is insufficient evidence in this study to conclude blue tongue is not caused by MMR?

    Now I do not dispute the fact that the Peltola studies may not have been methodologically robust enough to ascertain some neurological changes some months after MMR administration, but then that is not what parents claim happen to their children. Their stories often point to immediate, noticeable and significant alterations in behaviour. These are effects that the Peltola surveillance, which was extremely comprehensive, should have picked up over the background incidence. It didn’t.

    We are used to hearing claims that 1 in 100 children develop autism, and that MMR or other vaccines are the cause. In Finland, there should have been up to 15 thousand autism cases precipitated by the vaccine study. The most frequent causally linked side effect of the vaccine was fever in 0.07%, and ITP occured in 0.03%.

    The followup correspondence in the Lancet in 1998 from Peltola’s group looked at those who reported gastrointestinal symptoms after vaccination, and what the long term outcome was for these children. They found no evidence of autistic spectrum disorder in any of these children. The putative mechanism by which MMR is supposed to cause autism is through increased gastrointestinal permeability following inflammation of the gut from measles virus. It seems quite appropriate to conclude that these studies do not show a link between MMR and autism. In science, one can never prove a negative, as you know, but it should be clear to anyone looking at epidemiological studies like Peltola’s that even if MMR does cause autism, it does so in such a tiny proportion that it is indistinguishable from the background noise.

    I am not claiming MMR is “completely safe”, since it clearly is not and it can have adverse effects. However, I doubt precipitating autism is one of them, but I keep an open mind to the possibility that MMR may be a trigger in a tiny percentage of susceptible individuals, in the same way that the natural infection itself (measles) might be a trigger.

    As always, the relevant criterion by which to judge vaccination issues is to look at the risk/benefit equation. For MMR, this is heavily skewed in the favour of vaccination.

  12. Angie 2 June 2008 at 12:00 pm #

    One of the odd things about the whole debate in relation to vaccination is the fact that, for my generation (immediate post-WWII babies), measles, mumps and rubella were regarded as relativel unproblematic, routine childhood diseases. I had the first two as a child, as did most of the other children I knew, spending my time during the measles jumping up and down on the beds with my sister (I had rubella as an adult, which apart from avoiding pregnant friends and acquaintances, was entirely trivial). Overall they weren’t perceived as particularly dangerous diseases (unlike, say, diptheria). Maybe this had something to do with populations who were routinely exposed to these pathogens as against the ‘virgin’ populations now? (I can conceive of several ways in which the immune responses might have been different in that context.) This isn’t to say that no children were damaged by these diseases, but it was very very far from the scare stories we hear now about the consequences of a lack of vaccination.

  13. Dr John Briffa 2 June 2008 at 12:21 pm #

    David

    “I am not claiming MMR is “completely safe”, since it clearly is not and it can have adverse effects. However, I doubt precipitating autism is one of them, but I keep an open mind to the possibility that MMR may be a trigger in a tiny percentage of susceptible individuals, in the same way that the natural infection itself (measles) might be a trigger.”

    How refreshing (honestly and not in an ironic way) to find someone who brings what looks like an open mind to the arena. I agree with your sentiments quoted here, but I’m not sure about the ‘tiny percentage’ bit. I don’t claim to know any more than you. I find the Honda study critique linked to in my post no 54 quite concerning, for instance. I do think scientists should be looking at this to see if the data stacks up.

  14. jdc 2 June 2008 at 3:53 pm #

    There is more discussion of some of Dr Briffa’s points here: http://jdc325.wordpress.com/2008/05/30/more-briffa/

  15. Dr John Briffa 2 June 2008 at 6:27 pm #

    Re: comment 65:

    The author of the apathysketchpad blog, you will discover, a one Andrew Taylor.
    The link that he posts to above goes to a blog post, followed by a discussion he and I had on-line about a number of matters, mainly logic and science. Do, please read the posts, I urge you, because they aptly demonstrate lack of understanding of either of these two disciplines that I’m finding depressingly common in those who purport to be ‘scientists’. See him also demonstrate his very tenuous grasp of the science in the area of MMR and autism. I think, these exchanges demonstrate this particular brand of ignorance so well, that I’m linking to it again should anyone miss it. See here: http://www.apathysketchpad.com/blog/2008/05/30/a-briffas-wrong/

    Now, something you need to know: Andrew Taylor has a dim attitude of parents who believe their child’s autism may have been caused by MMR vaccination. He takes delight, it seems, in belittling such parents. With no proof that their belief is incorrect, he still has no hesitation in pouring scorn on them. How lovely.

    Now, in his defense, Andrew is a scientist, so we should take his word therefore for why parents who believe that MMR vaccination may have caused their child’s autism are deluded and need setting straight through chastisement. You see, expert Andrew is PhD student in 2D and 3D image analysis. How comforting that those who have autistic children to know that they can look to someone so aptly qualified for enlightenment and correction.
    .
    I’m warning you, Andrew Taylor’s site is not for those of a sensitive or caring disposition

  16. jdc 3 June 2008 at 11:16 am #

    “I’m warning you, Andrew Taylor’s site is not for those of a sensitive or caring disposition”
    Dr Briffa likes warning people. His warnings aren’t always so friendly though – he has threatened legal action against several people who commented on my blog.

    Here, John makes the accusations: http://jdc325.wordpress.com/2008/05/30/more-briffa/#comment-1070

    Here, he is provided with evidence of his deletions: http://jdc325.wordpress.com/2008/05/30/more-briffa/#comment-1130

    Will you “hold your hands up” now John?

  17. Dr John Briffa 3 June 2008 at 11:26 am #

    jdc

    I maintain that I have NEVER deleted posts. Now, please permit me some time to check the evidence, and get back to you. It may take a day or two, because I am seeing patients this afternoon and lecturing tomorrow. But I will look into it. Would that be OK?

    In the meantime, you might like to get back to me about those questions I pose to you here, and well as the question posed to you this morning in an email about why you protect your identity.

  18. jdc 3 June 2008 at 12:44 pm #

    I thought I had dealt with the substance of your arguments in my blog post. I do not particularly wish to answer questions about my personal circumstances on either your blog or mine (I think it can often be very boring to do so, I don’t think it can tell you anything useful and frankly I think the discussion of evidence is far more interesting), but I hope I can satisfy your curiosity about my personal situation via email. Black Triangle, Left Brain/Right Brain and Apathy Sketchpad have also answered some of the points you made. For instance, I do not intend to respond on the point you made regarding the Poling case as LB/RB has already done so.

    Perhaps you could highlight here the substantive points in your posts that you feel I have not yet answered?

    Re deleted/unapproved posts – yes, I would be grateful if you would look into this. Perhaps if you explained how you moderate this blog it would be possible to clarify that particular point now?

  19. MinorityReport 4 June 2008 at 3:07 pm #

    Re censored posts, try posting about Goldacre’s conflict of interests. Goldacre writes in the Guardian as if he’s an independent journo, debunking concerns about mobile phones and mmr, and calling people who disagree with him morons. The Guardian hides the fact that he works at the Institute of Psychiatry, where they are funded to produce industry and government propaganda disguised as research.
    http://www.cspinet.org/integrity/watch/200709241.html#3

  20. Sian 6 June 2008 at 5:03 pm #

    I can scarcely believe I have read right to the end of this debate – packed, as it is, with long words which fly right over the top of my blonde head…my question is this:
    Is the ultimate goal of the ever increasing number of baby vaccinations that no child will ever die of a childhood illness? Why are we brainwashed into being terrified of diseases that were a normal part of childhood when I was young.
    Like Angie in the post above, I had measles, mumps and german measles – I got some time off school and lots of jelly. WHAT IS THE PROBLEM? I dont know of anyone who suffered any long term problems from any of the diseases we all caught and gave to each other.
    As a mum of 4 children, I wouldnt like any of them to die or become permanently damaged by measles. Equally I wouldnt like them to fall under the wheels of a bus, be stabbed or beaten to death by a mob or become heroin addicts. But I have to accept the possibility that any of these things could happen.
    Maybe my children will live long enough to suffer a few years of neglect and dementia in a retirement home, perhaps they will die young. Death happens – we might as well get used to it.
    The one thing I DO have proof of: my unvaccinated children are far far healthier than my vaccinated children.

  21. jdc 11 June 2008 at 4:39 pm #

    “I maintain that I have NEVER deleted posts. Now, please permit me some time to check the evidence, and get back to you. It may take a day or two, because I am seeing patients this afternoon and lecturing tomorrow. But I will look into it. Would that be OK?”

    [3rd June 2008 11.26am]

  22. Dr John Briffa 12 June 2008 at 8:08 am #

    jdc

    The claim is I’ve deleted comments. Yet, the ‘evidence’ suggests that comments that were not there at one point are there now.

    In other words, this is not evidence of comment deletion at all.

    It looks to me to be the usual diversion from the true issues. Some dietitians use these tactics a lot, it seems. And they do their profession a massive disservice when they do, I think.

  23. ross 13 June 2008 at 11:24 am #

    Spongebob, comment 59. Do you have any evidence for the comments you make here:

    But if you persist (on his site) he will block your posts, as happens frequently there. Also happens on other sites such as LeftBrain/No Brain, Goldacres forum, Orac etc. They make sweeping statements, and to prove that these are true they prevent any dissenting voices from posting comments, and hey presto – no dissent = must be true.

    Dr B – so what was the reason for the magic reappearance of the missing comments? Were they deleted and re-instated or was there a very long comment moderation period?

  24. Richard 24 June 2008 at 3:56 pm #

    Some sense at last from Sian post No:71. I too have read this post to the end (phew!) and like the road accident discussed, it has been hard to not look and see the impending mess.
    I have three unvaccinated boys 3, 9 & 11 years old. We have only had one trip to the doctor out of all those years, never used antibiotics, do not use fluoride in toothpastes and none of them have fillings. This is replicated by all other non-vaccination families I know and am friends with. They are the healthiest kids I know and illness is seen as something important to the developing immune system (mumps was no bother). Other then non vaccination, no antibiotics (oral and household sprays), no fluoride and reduced chemicals (esp artificial sweeteners) and toxins in food and house, I can see no other things that we do differently to other friends. Why are the majority of my neighbours’ kids regularly very ill, on continuous rounds of antibiotics, steroids, suffering from asthma, autism, allergies. It may not all lie at the feet of vaccination but it is just another assault on the immune system.

    One question though – Siam maybe you have an idea? Now that these diseases are harder to find, my kids have not had measles and I am not sure if I want to risk them having it during teenage due to the increased dangers of getting the disease naturally but later than normal, so should I have the oldest one vaccinated by single dose with Measles (they have had mumps naturally – very easily). Can’t decide what to do? It is healthier to get the disease naturally but may be risky to get it as a teenager or adult?

Trackbacks/Pingbacks

  1. Apathy Sketchpad » Blog Archive » A Briffa’s Wrong - 2 June 2008

    [...] other day I posted about Dr John Briffa’s rant against p-values. He has since then posted some responses, in the form of several comments under the original post [...]

  2. AltMed Support for Wakefield Continues « jdc325’s Weblog - 15 March 2009

    [...] Dr John Briffa has also indulged in some MMR scaremongering of his own (Briffa’s original post), which I covered here, here, and here [note: the first of my blog posts includes links to many [...]

  3. Adverse effects of drugs are “neglected, restricted, distorted and silenced” | Hard Sweat - 17 February 2010

    [...] Dr Ioannidis also makes the point some trials limit their reporting of adverse effects to ‘statistically significant’ results. The problem is, the most serious adverse effects are generally very rare, and studies may not be large (powered) enough to define serious adverse effects as ‘statistically significant’. Dr Ioannidis claims that the statements such as “no significant differences were found” are misleading. I made a similar point some time ago in the blog post here. [...]

Leave a Reply