When testing the effects of different diets, it helps to ensure that the diets are truly different

Over the last few years there has been intense debate in nutrition circles about what the ideal diet is for weight loss. There are lots of different dietary approaches that may be advocated, but the most contentious debate has been about whether a diet low in fat, or one low is carbohydrate, is best if the goal is weight loss. There are now quite a few studies that have pitted low fat against low carb regimes. I’m not aware of any that show low fat to be superior in terms of weight loss. There are, however, several studies which have found low carb outperformed low fat.

Some of you may have read about a study published last week which was published in the New England Journal of Medicine that is said to prove that for weight loss, it’s not dietary composition, but merely calorie intake, that’s important. The study tested four diets of varying composition over a 2-year period. The diets varied in terms of their content of fat, protein and carbohydrate [1].

The percentage if calories contributed by fat/protein/carbohydrate for the four diets was as follows:

Diet 1. 20/15/65
Diet 2. 20/25/55
Diet 3. 40/15/45
Diet 4 40/25/35

At the end of 2 years, there was no significant difference in weight loss between the groups, and average weight loss was 4 kg (9 lbs). The conclusion of this study is that Reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize. This study has been used to vindicate the conventional line that a calorie is a calorie: it doesn’t matter what form calories take, their impact on weight is the same.

But is this really so. For start, while study investigators and their assistants may prescribe specific diets to the participants, there is no assurance that they’ll stick to them. Compliance issues are common in dietary study, and so the results of such an experiment are ideally assessed not on the basis of the prescribed diet, but the diets that were actually eaten.

Here is the actual nutritional make-up of the diets eaten by the individuals in the four groups when they were assessed at the end of the study (percentages have been rounded to nearest whole number):

Diet 1. 27/20/53
Diet 2. 28/21/51
Diet 3. 33/20/49
Diet 4. 35/21/43

It’s clear from these figures that the diets were, overall, much more similar in make-up than the prescribed diets. For example, the percentage of calories contributed by carbohydrate in diets 1 and 4 should have been 30 per cent (63 minus 35 percent). In reality, the difference was 10 per cent. Those eating diet 1 should have been eating almost twice as much carbohydrate as those eating diet 4. In reality, they ended up eating 23 per cent more carb.

Because the diets were so similar in composition, it seems inappropriate to use it to vindicate the ‘calorie is a calorie’ concept.

And what this study certainly can’t be used for (though some have tried) is to suggest that low-carb diets have no particular merit.

One reason for this is that diet 4 – the ‘low-carb’ diet ” was most certainly not low in carb. Individuals in this group were getting about 43 per cent of their calories from carb. Let’s for argument sake say a low-carb diet contains no more than 40 g of carb a day (the induction phase of the Atkins diet allows just half this amount). This 40 g equates to about 10 per cent of calorie intake. That’s less than a quarter of the percentage of calories contributed by carb in diet 4.

Also, those eating diet 4 were consuming, on average, 152 grams of carbohydrate a day (almost four times as much as someone eating a true low-carb diet containing 40 g of carb a day).

While the authors of this study mention low-carb diets several times, it seems they were not necessarily keen to test the effectiveness of such a diet. If they had been, then they would have done well to include a diet that was actually low in carb.

The other thing to consider is that while weight loss may indeed be a laudable aim, it is perhaps worthwhile considering that different diets have been shown to have different effects on biomarkers of illness and health including blood fat levels. Here, low-carb diets have been found to be generally superior to low fat ones .

Also, there is emerging evidence that protein-rich diets can promote weight loss in a way that cannot be explained by mere calories. A year ago I blogged about a study which pitted a high protein diet against a high carb one in women. The effects on weight were essentially the same. But, as is often the way, there had been compliance issues in this study. Analysis revealed that the more protein women ate, the more weight they lost. If you take a look at the figures from the NEJM study again, you’ll see that individuals in the four diet groups ended up eating, essentially, the same amount of protein (20/21 per cent of calories in all four groups).

So, my advice to the authors of the recent NEJM study is this: if you want to test the effect of different diets on weight or any other aspect of health, then it helps to make sure that these diets are truly different.

References:

1. Sacks FM, et al Comparison of Weight-Loss Diets with Different Compositions of Fat, Protein, and Carbohydrates. NEJM 9;360:859-873

18 Responses to When testing the effects of different diets, it helps to ensure that the diets are truly different

  1. Peter A 4 March 2009 at 7:07 pm #

    It is intersting that, just like the Framlington Study, this study has found that in America in particular it seems is very difficult to get people to stick to a low carb diet.
    I remember a study done some years ago in the UK (by a TV program) that looked at people’s shopping and analysing the food that they brought. From memory the group on the high protein diet lost the most weight.

  2. Chris 4 March 2009 at 9:49 pm #

    Hi John,
    and did those behind this study distinguish the sources of the carbs?

  3. Sue 5 March 2009 at 10:46 am #

    What a total waste of money!

  4. Neil 5 March 2009 at 9:45 pm #

    ” It is intersting that, just like the Framlington Study, this study has found that in America in particular it seems is very difficult to get people to stick to a low carb diet.”

    Perhaps this has something to do with the popular ‘wisdom’ that a) people don’t stick to low carb because its boring b) low carb makes you feel grotty.

    As Sue says, what a waste of money and opportunity

  5. The other (non-dietician)Kate 6 March 2009 at 2:58 pm #

    One day, someone, somewhere will do a diet comparison that actually includes a ‘proper’ low-carb diet.
    It is so frustrating to read this sort of study over and over again.
    The only way to change the carbs-first mindset, is to recruit people who understand the importance of testing the diet and who have the emotional intelligence to stick to it.

    This is well-nigh impossible and very disappointing. The longer I follow this dietary approach, the more I encounter people who just don’t ‘get’ it.

  6. Hilda Glickman 6 March 2009 at 10:40 pm #

    43% from carbs. THis is certainly not low carb. What are they thinking about? So when anyone talks about low or high carb and its effects they need to specify exactly what they mean.

  7. Tony 8 March 2009 at 12:42 am #

    But doesn’t this study test the ability of the participants to stick to their assigned diets and, ergo, whether the diets were succesful?

    http://content.nejm.org/cgi/content/full/360/9/859

    So isn’t looking at the macronutirient content of the make-up of the diets at the end of the study irrelevant?

    Perhaps Doctor Briffa could explain why the conclusion of study is invalid?

    Reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize.

  8. Tony 8 March 2009 at 12:48 am #

    Here’s a good overview of the study:

    http://www.nhs.uk/news/2009/02February/Pages/Dietsweighedup.aspx

    It states:

    Sticking to diets can be difficult and the participants in this study were supported by regular counselling sessions. Although the low carbohydrate diet in this trial had a target of a maximum 35% carbohydrates, most people did not achieve this.

  9. Dr John Briffa 8 March 2009 at 5:49 pm #

    Tony

    “So isn’t looking at the macronutirient content of the make-up of the diets at the end of the study irrelevant?”

    No, not if you’re (as I and some others are) interested in the relative merits of diets of genuinely different macronutrient make-up.

    “Perhaps Doctor Briffa could explain why the conclusion of study is invalid?

    Reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize.”

    But where did I state that this conclusion is invalid?

  10. Trinkwasser 8 March 2009 at 7:13 pm #

    Everyone and his dog, er blog, has been shredding this study, and justifiably so.

    IMO MIchael Eades did the best job

    http://www.proteinpower.com/drmike/bogus-studies/last-gasp-of-the-dark-ages-of-nutrition/

    but everyone else was a close runner up

  11. Gordon 8 March 2009 at 9:33 pm #

    I believe the theory of the eminent nutritionist Dr Mercola on the question of what is the “best” diet may be relevant here. He says there are three dietary “types”: Protein, Carbohydrate and Mixed. There is no “one-diet-fits-all” diet. To eat the best diet for you depends on eating a diet suited to your type. It is not difficult to discover what that is. If experiments like the one Dr Briffa discusses do not take dietary type into account, the results are bound to be unreliable. I recommend anyone interested to visit the Mercola website

  12. Tony 10 March 2009 at 3:24 am #

    “So isn’t looking at the macronutirient content of the make-up of the diets at the end of the study irrelevant?”

    No, not if you’re (as I and some others are) interested in the relative merits of diets of genuinely different macronutrient make-up.

    Can I suggest that you are looking in the wrong place then and are criticising this study because it’s not what you want it to be?

    So, my advice to the authors of the recent NEJM study is this: if you want to test the effect of different diets on weight or any other aspect of health, then it helps to make sure that these diets are truly different.

    But if you want, like the study authors, to test the success of the diets, then you’d probably use the same methodolgy that they do.

    Perhaps Doctor Briffa could explain why the conclusion of study is invalid?

    “But where did I state that this conclusion is invalid?”

    So you agree that “reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize”. Do you have any meaningful criticism of what the authors set out to study and their methodology?

  13. Dr John Briffa 10 March 2009 at 1:37 pm #

    “Can I suggest that you are looking in the wrong place then and are criticising this study because it’s not what you want it to be?”

    The study was designed to test the effects of diets of different macronutrient composition on weight. But it was not fit for purpose (see above).

    Tony

    “But if you want, like the study authors, to test the success of the diets, then you’d probably use the same methodolgy that they do.”

    Actually, no: I would have included at least one group randomised to a true low-carbohydrate diet.

    “Do you have any meaningful criticism of what the authors set out to study and their methodology?”

    Yes, see post above.

  14. Tony 10 March 2009 at 3:25 pm #

    The study was designed to test the effects of diets of different macronutrient composition on weight.

    So not the effects and thesuccess or otherwise of the diets then?

    You will also note that the conclusion states that “reduced-calorie diets result in clinically meaningful weight loss regardless of which macronutrients they emphasize

    If they had set out to study the effects of the macronutrients in the diets, they would have excluded the data from subjects that were unable to stick to the diets.

    Actually, no: I would have included at least one group randomised to a true low-carbohydrate diet.

    Therefore your study would have set out to examine something completely different to that which the authors of the study set out to examine.

    Do you have any meaningful criticisms of the study?

  15. Sue 23 March 2009 at 6:27 am #

    Tony, read the study properly and then you might understand.
    Stop criticising just for the sake of it.

  16. Tony 24 March 2009 at 9:37 am #

    I made some informed comments Sue. If you have read the study properly would you care to explain why my comments on Dr Briffa’s piece are incorrect?

    Or are you just criticising for the sake of it?

Trackbacks/Pingbacks

  1. Are all Diets created Equal? It’s just about calories only afterall? | Health Tips - 7 March 2009

    [...] a recent study reviewed by Dr. Briffa they maintain all diets are equal and that it is after all only calories that matter, no matter [...]

  2. Calorie deficit vs macronutrient ratios - Page 4 - Myprotein Forum - 9 March 2009

    [...] carb diet still gave better results even with such a small difference is perhaps more significant. Source I’m even more amazed at how such a bad experiment design and conclusion can proclaim such results [...]

Leave a Reply