Why epidemiological studies don’t ‘prove’ anything

January is typically a time when individuals seek to adopt healthier habits, and activity and exercise often feature here as part of the plan. While it is often said that exercise is healthy, actually there is not as much evidence for this as you might think.

The best test for any lifestyle factor is to subject people to it, and compare the effects to those achieved in individuals who were not subjected to the same change. If weight loss is the aim, the results of such ‘intervention’ studies show that exercise is not very helpful at all [1-3]. I have previously written about why exercise generally has limited impact on body weight, and what might work better for those seeking to lose weight [4]. Also, weight is only one aspect of health. It may be the case that activity and exercise have some capacity to protect against potential conditions such diabetes and heart disease.

Even here, though, the evidence is not as robust as you might imagine. Conditions like heart disease and type 2 diabetes generally develop over many, many years. This means that intervention studies assessing the role of exercise in the prevention of heart disease may need to go on for years, possibly decades, to show any meaningful effect. Such studies are costly and impractical, and so evidence of this type is scant.

In the absence of appropriate intervention studies, we may look to what is known as ‘epidemiological’ data. In short, such research looks at associations between a factor (in this case, exercise) and disease risk. Such studies generally find that those who are physically active are usually at a reduced risk of conditions such as diabetes and heart disease. Ideally, these observations should be made over a period of time. Such studies are described as ‘prospective’ in nature.

The results of these studies can be weakened, however, by the fact that physically-active individuals may also, compared to sedentary individuals, have other factors in their favour. Active individuals may, for instance, be less likely to smoke and have lower body weight than less active individuals. Such factors may be the real reason for the apparent reduction in risk of ill-health seen in active individuals.

It is important, therefore, in epidemiological studies to take into account these so-called ‘confounding factors’. While the process of controlling for confounding factors is never perfect, it’s generally a damn sight better than not doing it at all.

Confounding factors were considered in a recent study which explored the relationship between physical activity and heart disease in British men and women [5]. The researchers took into account several confounding factors including smoking habits, weight, blood pressure, and the presence of diabetes. Having done this, the results showed that an active lifestyle was associated with a 32 and 49 per cent reduced risk of heart disease in men and women respectively. This study, like others, shows that activity and exercise is associated with protection from heart disease.

Notice the words I have used here: ‘is associated with‘. I’ve used these words because epidemiological studies cannot be used to prove that one thing causes another. For that, we’d require positive results from those rather elusive intervention studies.

Now, bear this in mind now as you know take a look at the title of this study. The suggestion implicit in Leading an active lifestyle substantially lowers the risk of future coronary heart disease in apparently healthy men and women is that the researchers have proven that exercise leads to a reduced risk of heart disease. And that’s simply not the case.

Now, I’ve noticed that lots of news reports seem to suggest that lifestyle factors have ‘proven’ health effects, despite the fact that they are based on epidemiological evidence ” Housework cuts risk of breast cancer” is one recent example that springs to mind. I suppose one could just about forgive the journalists that churn out such headlines without thinking. I don’t find it quite as easy, however, to excuse scientists and researchers who are responsible for the epidemiological studies themselves.

It is beyond me that the authors of the recent epidemiological study on exercise and heart disease do not realise that the title they have come up with simply cannot be justified from a scientific perspective. And what has happened to the ‘peer review’ process here, where other scientists get to comment on and correct inaccuracies in a piece of science?

What may have happened here, of course, is that the title was chosen for maximum impact, at the expense of accuracy. If researchers and scientific journals cannot be relied upon to provide accurate interpretation of whatever evidence they provide, no wonder increasing numbers of individuals are growing sceptical about the ability of the holy church of science discern fact from fiction.

References:

1. Reilly JJ, et al. Physical activity to prevent obesity in young children: cluster randomised controlled trial. BMJ, doi:10.1136/bmj.38979.623773.55 (published 6 October 2006)

2. Votruba SB, The role of exercise in the treatment of obesity. Nutrition. 2000 Mar;16(3):179-88

3. Shaw K, et al Exercise for overweight or obesity. The Cochrane Database of Systematic Reviews. 2006 Issue 4

4. Why exercise is not a cure for obesity and what works better for those seeking to shed weight in the long term

5. Matthijs Boekholdt S, et al. Leading an active lifestyle substantially lowers the risk of future coronary heart disease in apparently healthy men and women. European Journal of Cardiovascular Prevention and Rehabilitation. 2006;13(6):970-976

2 Responses to Why epidemiological studies don’t ‘prove’ anything

  1. helen 18 January 2007 at 11:06 pm #

    I find this type of sensational headline for articles about health pretty ugly. Then when all you are advertising is your product to increase the profit of your company, well why wouldn’t headlines like this appear? but to actually have scientists make up these types of headings for their study reports is even more evil than the papers who then wish to create more sensation. Speaking in absolutes from so called “experts” is all the rage because hey if an expert says so then it must be true – especially if I read it in the papers or hear it on TV!!! Epidemiological studies may not prove anything but they can go a long way to backing up “real” scientific study (does that exist anymore & is it always relevant when dealing with individuals?)
    I think the fault lies more with the manipulation of data & twisted spin doctoring for advertising of products rather than the study of data observed & collected.

  2. Ian Hamilton 19 January 2007 at 4:03 pm #

    On reading your latest excellent blog about “Why epidemiological studies don’t ‘prove’ anything” I followed a link to “Why exercise is not a cure for obesity and what works better for those seeking to shed weight in the long term”

    I posted the questions below as a comment but the thought occurs to be that as it is an archived post, you may not have seen it:

    It seems very reasonable that exercise has no effect on “weight” however I have a question:

    We are always told (but is it fair to say?) muscle is denser than fat. It is a reasonable (&testable) hypothesis that regular exercise would increase muscle mass/reduce fat mass.
    So even if your overall weight stays the same, regular exercise could help to reduce the amount of fat you carry.

    I would definitely take the point that for anyone who is already obese overall weight reduction is probably an order of magnitude more important than lowering their fat to muscle ratio, but by obese don’t we really mean “carrying too much fat”. Ergo even in these extreme cases weight reduction is really fat reduction which exercise would actually help with?
    People aren’t obese because they have too much muscle

    I don’t think this is merely a pedantic point. I appreciate you have already recognised the benefits of exercise for preventing some specific things e.g. heart disease and I totally agree that exercise role is limited in weight loss. I guess I am actually saying that “fat loss” should be the focus not “weight loss”, which I don’t think you wld disagree with at all – maybe?

Leave a Reply