While many doctors and health professionals like to believe they practice ‘evidence-based’ medicine, the reality is that much of healthcare practice is simply not supported by good science. This isn’t necessarily a problem, of course, as there are other means that practitioners might use to base their practice decisions. One of these, as I pointed out recently, is clinical experience. Unfortunately, some proponents of ‘evidence-based medicine’ seem to have forgotten this component. Though, I have to say my experience has been that such individuals are usually academically-minded, and don’t actually see patients. This might help to explain why some of these individuals seem not to value clinical experience (they don’t have much or perhaps any at all).
However, there are other reasons why it does not necessarily make sense to base clinical decisions on science: the science can be unreliable. It appears that the conclusions study authors come to can be influence by the source of funding. For example, I previously reported on evidence which found that for blood pressure lowering medications, drug company funded studies were 4-5 times more likely to report positive findings compared to those that had been independently funded.
What this study found was that results from industry-funded and non-industry funded studies was essentially the same. It was the way in which the results were interpreted and reported that was different. Such so-called bias can give a very distorted picture regarding the effectiveness of a treatment, particularly when busy doctors feel they only have time to read the conclusion part of the summary of an article (known as the abstract).
The authors of a study published in this week’s British Medical Journal make this point as they set about determining whether bias is a factor in the publication of studies relating to flu vaccination. They chose flu vaccination as their subject partly because this practice is advocated on a near-global basis. Yet, as they point out, the evidence supporting this intervention is generally poor in quality and contradictory in nature. Not so long ago I reported on a study which calls into question the veracity of the oft-quoted ‘fact’ that flu vaccination reduces mortality in the elderly.
Having reviewed the literature in the area, the authors of the BMJ review concluded that (as with studies on blood pressure lowering medication) there is often a discrepancy between the results of a study and the conclusions drawn by the authors. However, here, there did not seem to be any relationship between the source of the funding and the discrepancy. In other words, industry-funded studies were no more likely to report inappropriately positive conclusions than other studies. However, as the authors of the review point out, almost a quarter of the studies they assessed had no declared source of funding, and without the full facts it’s not possible to know for sure if source of funding led to bias in terms of how the results were interpreted and reported.
What the results of this review also showed was that most (70 per cent) of the studies were of poor quality and drew generally positive conclusions simply not supported by the data. Industry-funded studies, though no better in quality, were more likely to be published in prestigious medical journals and therefore had greater visibility.
The authors of the review state that they cannot say for certain why industry-funded studies are seemingly more attractive to the more prestigious journals. However, it seems they are suspicious that money might have something to do with it when the recommend that As a measure of transparency for readers and authors we recommend that once a year editors and publishers should post all sources of income relating to the running of the journal.
The authors end their review by pointing again to the discordance between study results and conclusions. In most cases, what you see is not necessarily what you get, the authors lament. There is reason to believe that flu vaccination policy may be based on misreporting and misrepresentation of the evidence. In other words, it may not be based on the evidence at all.
References:
Jefferson T, et al. Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review. BMJ 2009;338:b354, doi: 10.1136/bmj.b354
Dr John, It was through reading Patrick Holford a number of years ago that I first began to question the notion of a ‘pill for every ill’. I declare myself to be a layman with a limited knowledge on the subject of medicine but the argument for fully considering the alternatives before administering compounds alien to our evolutionary heritage would seem to make good sense to me. (My point is a general one so don’t challenge my limited knowledge on vaccines and anti-bodies!)
OK, the health service has taken steps to more preventionist policies; evident to me in that my GP nags me about lifestyle factors in a number of consultations before resorting to the prescription of statins.
Despite the preventionist policies, despite that my GP can prescribe discounted visits to my local gym, despite the growing knowledge in relation to food, nutrition and health, why do obesity rates continue to rise? Why do T2 diabetes rates accelerate so astronomically?
The health service was so long too dependent on the prescription culture and thankfully has made noticeable moves away from that. A friend of mine, himself a former GP, who traverses the world in his employment with Astra Zeneca told me recently,”It’s not so easy now, the Klondyke days are over.”
At least in relation to prescription, then, things have changed for the better.
As you so rightly point out, Dr John, there still exists an enormous lack of impartiality in the science that leads us to decision making.
I long harboured concerns about over-powerful drug companies tipping the field in their favour. You would think that the resources and intellect within our health service professionals would notice the field tipping. Nonetheless, evidently bias still slips in and becomes embedded.
In the health service the work of the professionals does at least offer a ‘buffer’ between the dark forces seeking to exploit them and the patient.
With food, I have become increasingly alarmed at the lack of protection for the consumer. Instead, consumers are fed a diet of official advice of which some may be based on ‘bad’ science, be unnecessarily confusing, burdening, or just plain daft.
OK nobody is perfect, and not least the Food Standards Agency, but the bewildering official information presents the ideal environment of confusion in which the food industry can add to the misinformation by the way of advertising; and then, of course, exploit the poor consumer in the interests of profit and to the detriment of the health of the consumer.
It’s pretty plain where we’ve gone wrong. Because of very little unbiased funding in relation to research we lent our destiny to hands of people who have every interest in profit and limited interest in our well-being.
That is as true when you buy your ready-meal from the chiller or freezer at the supermarket or whether you want to make an ‘evidence based’ decision in relation to prescribing.
There is little profit to commerce in ‘thin’ and/or ‘healthy’
On the other hand there would be great financial benefit to GB plc if the poeples of the nation were both thin and healthy.
Do politicians number among the readers?
John, a minor point; I notice the time-stamp is wrong!
I have 16:13
I have to agree with Agent 3244 although I have to slightly disagree with his comment
“In the health service the work of the professionals does at least offer a ‘buffer’ between the dark forces seeking to exploit them and the patient.
Unfortunately many of the professionals are too busy ‘fighting in the trenches’ in trying to treat people to read all the literature and only have time to scan the abstracts. For example here is a belief in my local surgery that statins are safe. Having found an original drug company report I took hte trouble to read it. In the report there are are about 3 pages of the known side effects burried in a 30 or 40 page report.
The abstract says that this statin is better in higher doses.
It did not mention the problems it causes and also doesn’t bother to mention that statins have have no effect on morbidity.