Scientific evidence appears to be becoming steadily more unreliable

I am a relative fan of science. I say ‘relative’ because while it can often afford considerable illumination, it’s most certainly not the be-all-and-end all. For example, there are plenty of things that may have merit that have not even been subjected to proper scientific study. For example, there are no studies (in the form of ‘randomised controlled trials’) that conclusively prove stopping smoking is beneficial to health, but that wouldn’t stop me from suggesting that someone stop smoking if they asked for my advice on the matter.

Also, even when something has been subjected to systematic scientific study, the evidence base can actually give a very skewed version of reality. One way this can happen is as a result of what is known as ‘publication bias’.

Imagine there’s a widely held belief that, say, saturated fat causes heart disease. Studies that support this idea are viewed as ‘positive’ studies, while those that don’t are ‘negative’. There can be a tendency for medical and scientific journals to preferentially publish positive studies. In other words, studies that are in line with current thinking are more likely to make their way into the scientific literature than negative ones. In this way, existing dogma can essentially go unchallenged – something that is inherently unscientific.

The problem can be compounded by scientists themselves.

It’s worth bearing in mind that researchers gain from publication in two main ways. Firstly, it can enhance their reputation and standing in the scientific community. Also, the most published researchers are the ones most likely to attract funding for their work. For some scientists, regular publishing is a matter of professional survival.

Researchers can work out for themselves the sorts of studies that stand the best chance of being published. There can be, therefore, an innate push among scientists to produce research which does little or nothing to further our understanding of a particular subject.

Recently, researchers from the University of Edinburgh in the UK published an analysis of 4,600 research papers from a wide range of disciplines including clinical medicine, psychology, psychiatry and pharmacology. The research was published from 1990 to 2007. What the researchers noted was a steady decline in the proportion of negative studies over thie time period. In 1990, some 30 per cent of studies were negative, but by 2007 this figure had dwindled to just 14 per cent.

There is evidence, therefore, of an increasing tendency for the scientific community to churn out ‘more of the same’. The pressures on researchers and scientists to produce ‘meaningful’ results can ultimately lead to us getting a very biased view on reality. On the outside, an increasing consistency of positive results can look like the evidence is becoming ever more reliable. In reality, though, the evidence may in fact be getting steadily less reliable.

This is just one example why science is not always to be trusted implicitly, and why those who stick slavishly to it are usually not to be trusted either.

References:

1. Daniele Fanelli. Negative results are disappearing from most disciplines and countries. Scientometrics epub 11 September 2011

5 Responses to Scientific evidence appears to be becoming steadily more unreliable

  1. Michelle Berriedale-Johnson 16 September 2011 at 6:57 pm #

    All too depressingly true…. Those interested in this subject might also like to read a report of the British Society for Ecological Medicine’s excellent conference last year: A tablet a day keeps the patient at bay: the health hazards of prevention – vaccinations and pharmacoprophylaxis to be found at http://www.foodsmatter.com/conference_reports/articles/bsem_2011_tablet_a_day.html
    See especially the first presentation by Dr Jayne Donegan.

  2. Paul 17 September 2011 at 1:22 am #

    An, hence, why the US is getting more obese and diabetic whilst the science seems to say more of the same.

  3. Jane lord 17 September 2011 at 6:24 pm #

    This is of course compounded by the fraudulent practice of ‘authoring’ studies , when the named authors have nothing to do with actually producing the research or the paper. Just a name, as well-known as possible, to earn a great deal in fees….

  4. Joseph Putnoki 18 September 2011 at 8:51 pm #

    Again, well said Dr John!
    My close to middle age doctor is a good person but captured by the industry. I hired him some 8 years ago and found him open for learning. Once he said doctors also learn from patients. So I took articles to him the only request he made that to be from reputable sources. 2 years later things started to change and he became defensive and dismissive. Trying to educate him about pharma propaganda, practice guidelines and some that unlikely to cross his desk. Trying to be helpful he lent me 3 sleek brochures about summaries of a couple of clinical trials as evidence contrary to my concerns. One was the Jupiter trial and the conclusions of course positive. The truth how ever is different as I learned more investigating the matter, one example:
    http://www.proteinpower.com/drmike/cardiovascular-disease/1853/
    My doctor desperately trust the information from industry uncritically while maintain skepticism he days. Our exchanges became unproductive and time limits put paid to my further efforts. He could not believe things are possible what in fact is happening. Pity so many good-hearted doctors do harm while thinking doing good!

    Be well!

    joseph.

  5. Henrik 20 September 2011 at 2:02 am #

    Dear Dr. Briffa
    I think you should read these links.

    http://www.dietdoctor.com/kids-removed-from-family-because-of-obesity

    and
    http://www.dailymail.co.uk/news/article-2033486/Your-children-fat-again.html.

    I am a big fan of you and your podcasts.
    Best greetings from Denmark

Leave a Reply