There’s a study doing the rounds that is being reported as evidence that “Atkins was wrong”. Apparently, according to those reporting the study, it shows that eating more meat generally causes increased weight gain over time [1]. To see a typical way in which this study is being reported, see here.
Sitting on a plane yesterday I read this study in its entirety. And even cursory inspection of it reveals some things which some journalists and reporters may have missed.
The first, most obvious, thing to note is that the study is epidemiological in nature. It looked at a total of 16 female and 13 male populations over some years, and when all the results were lumped together there was, apparently, an association between meat eating and enhanced weight gain over time. Eating an additional 250 g of meat each day appeared to translate into an extra 2 kg over five years. The authors of the study state “This absolute increase may be considered low from a clinical point of view.”
But getting back to the epidemiological nature of the study, we know that, at best, all the we can infer from this study is that meat-eating is associated with weight gain. That does not meat it causes weight gain. Other so-called “confounding factors” (maybe big meat-eaters ate other foods that were “fattening”, and that’s the real explanation behind the finding, for instance). Now, the authors attempted to control for these “confounding factors”, but this is always an imprecise science. Certainly, there was no attempt in this study to consider the relationship of other major foods (e.g. bread, pasta) and weight gain.
Other problems associated with this study include the fact that diets were assessed with a questionnaire once (at the beginning of the study) and never again, and that weight was generally self-reported (and this can be prone to mis-reporting). The other thing, of course, is that weight tells us practically nothing about health. Body composition would be a much better judge here. Maybe, if meat really does lead to increased weight, this might be in the form of muscle, rather than fat. Who is to know? Well we can’t know, because this study looked at weight alone, and tells us nothing therefore about changes in body composition.
So, these are just some of this studies major deficiencies.
What other evidence might we look to then? Well, to really look dissect the truth about the impact of a food (e.g. meat) or macronutrient (e.g. protein, fat) on weight and/or health, we require intervention studies. Put people, say, on a high meat/protein diet, for instance, and see what happens. Well, studies show that such a diet, compared to a low-fat diet, generally leads to improvements in terms of weight loss and markers of cardiovascular disease and diabetes.
The authors of the study actually acknowledge the evidence showing that effectiveness of higher protein diets in weight loss, stating – “studies performed in overweight or obese subjects under energy restriction did observe a higher weight loss with a high-protein diet than with a high-carbohydrate diet.” They even list several studies that support this. The authors also concede that most studies show that when carbohydrate is partially replaced by meat, no weight change occurs.
Yet, even after these acknowledgements, the authors go on later to declare that “[More] importantly, our results do not support that a high-protein diet prevents obesity or promotes long-term weight loss, contrary to what has been advocated.”
What the authors seem to be saying here is that we should ignore abundant interventional (good) evidence that a meat/protein rich diet is good for weight control, and we take more note of their low quality epidemiological (basically, useless) evidence.
Also, while this study did find, overall, an association between higher meat eating and weight gain when all the results were lumped together, the results were highly “heterogenous”. This means, in this case, the results varied a lot between populations. This is particularly relevant, as when look between two populations (e.g. UK residents and those living in France) there is huge potential for confounding. When we look at a single population, then the risk of this is reduced.
Looking at single populations, the authors found a link between higher meat eating and weight gain in 6 populations. However, there was NO SUCH LINK in the remaining TEN populations studied.
And here’s another thing. There was evidence in certain populations that higher meat eating might actually protect against weight gain. And that lower meat consumption might promote weight gain.
In men, the population eating the most meat, actually had only the second highest gain in weight. The population eating the least meat, had the 10th highest weight gain (out of 13).
In women, the population eating the 12th highest amount of meat (Spain), actually had the lowest weight gain overall. Oh, and Danish women ate the most meat, but only one population enjoyed less weight gain than them.
You are at liberty, of course, to make of this study what you will. I’ll tell you what I make of it though: it’s not worth the paper it’s written on. Actually, worse than that, it appears to be wholly misleading.
References:
1. Vergnaud A-C, et al. Meat consumption and prospective weight change in participants of the EPIC-PANACEA study. Am J Clin Nutr 2010;92:398-497
Superb analysis John! I had to dedicate far too much time this week fending off this study and explaining to people the short-comings of this sort of epidemiological study. I was going to post on it, but decided it wasn’t worth it – will now just link to you.
I note in the study, in the regions that did show an association, the highest was with chicken. I immediately thought that I’d be more likely to eat chicken PASTA and enjoy a steak with some roast vegetables! Of course this study cannot tell me if the chicken eaters also enjoyed chicken pasta – because they didn’t account for carbohydrate intake.
This image sums these sorts of reports up nicely;
The Science-News Cycle
http://asymptotia.com/wp-images/2009/05/science_news_cycle.jpg
Great writeup, it confirmed my gut feeling that it was a completely bogus study. My instinct was that it had too many variables and was just cherry-picking stuff that the “researchers” felt important.
I love this comic, it’s quite subtle but should be required reading for all researchers:
http://xkcd.com/552/
Repeat after me: “Correlation does not imply causation”
They’re basically talking about 1 extra steak per day (example in the abstract) being responsible for 1 extra pound of weight gain per year. Rather silly and short-sighted. When you have to look that hard and come up with something, you’re conclusions are basically insignificant.
While I generally support your view of things I think you were a little unfair taking them to task for this statement:
“[More] importantly, our results do not support that a high-protein diet prevents obesity or promotes long-term weight loss, contrary to what has been advocated.”
Why do you berate them for this? They are specifically referring to their results – not opining about ‘reality’. Their study results, as interpreted by them (possibly incorrectly), don’t support those things.
You would have been right to criticize them if they said “Our study shows previous studies were wrong and therefore meat does not promote long-term weight loss”
Peter
Read the study as a whole, and the authors do seem very keen to dismiss high protein/low-carb diets on the basis of their weak evidence. And, as I point out, this is in the face of considerable (much better) evidence to the contrary. Looks like bias to me.
I know that one person’s experience does not constitute evidence, but at least my case is extremely well documented. I had a liver and kidney transplant nine years ago and have eight blood tests per year and regular scans of my transplanted organs. Since I cut my carb intake about eighteen months ago and increased my animal fat intake, my haemoglobin levels have increased and my creatinine has gone down. I feel better than I have done in years and my consultants are amazed at how well I’m doing. I can’t claim to be losing weight, but at least I’m no longer gaining weight, despite the immunosuppressant drugs I have to take. Oh yes, I haven’t had to take antibiotics for over a year now, and that’s practically unheard of for a transplant recipient.
And who says that weight is bad? If they’re eating meat, then it could be muscle mass.
I am afraid that this paper is just another poorly designed and poorly analysed paper that should never have been published except for the fact that it supports “official guidelines”.
There is no gain or other status benefit in opposing the KOLs (Key Opinion Leaders), only loss of jobs and ny status.
As Max Planck is reputed to have said “Science progresses FUNERAL by FUNERAL” I personally call it the Galileo Syndrome
Jamie,
I did like your cartoon ref. It is so true because most science journalists have not got a clue on assessing a research report
It’s such a waste of time and effort and money doing ‘research’ like this: faulty reasoning, incorrect conclusions, misleading, damaging advice. How/Why do researchers of such low calibre get funding?
Its all very frustrating http://ramblingsofacarnivore.blogspot.com/2010/07/frustrations-of-low-carber.html
Seems like these researchers learned their cherry picking skills from Ancel Keys and Colon Campbell…