By Marika Sboros
Did researchers at top South African universities make multiple mistakes in a major study deliberately? Was their aim to discredit low-carbohydrate, high-fat (LCHF) diets? And was their real target scientist Prof Tim Noakes? Or are they just human, fallible and in this case, hopelessly error-prone?
A new study in the SAMJ (South African Medical Journal) re-examines a 2014 study by University of Cape Town (UCT) and Stellenbosch University (SU) scientists. It shows that the scientists made many material errors that undermine their conclusions. It raises the question: mistake or mischief? Read on and make up your own mind.
The study in question is the “Naudé review” published in PLoS (Public Library of Science) One. Lead author is Dr Celeste Naudé, head of Stellenbosch University (SU) Centre for Evidence-Based Health Care. Co-authors include Prof Jimmy Volmink, head of SU Faculty of Healthy Sciences, and Prof Marjanne Senekal, head of UCT’s Human Nutrition Division.
Senekal is a consultant to the Health Professions Council of SA (HPCSA) in its ongoing trial of Noakes and LCHF.
Noakes and British obesity researcher Dr Zoë Harcombe are the authors of a re-examination of the Naudé review in the SAMJ. They say that the “pivotal importance” of the review in the HPCSA’s trial against Noakes prompted them to re-examine it. Click here to read: The universities of Stellenbosch/Cape Town low-carbohydrate diet review: Mistake or mischief?
Harcombe was an expert witness for Noakes. At the October session of the trial, she presented evidence on the many flaws in the Naudé review. She and Noakes say that the many errors leave the review fatally flawed.
The HPCSA has charged Noakes with unprofessional conduct for giving “unconventional advice to a breastfeeding mother on a social network”. That was a single tweet in February 2014, saying that good first foods for infant weaning are LCHF. Johannesburg dietitian Claire Julsing Strydom reported him to the HPCSA. Click here to read: The real beef dietitians have with Noakes.
Evidence at the fourth trial session in October 2016 shows that the HPCSA waited for the Naudé review before charging Noakes in September 2014. Evidence also shows that the HPCSA has based much of its case on the review.
The Naudé review authors concluded that low-carb diets are “no more effective for producing weight loss than are high-carbohydrate, or so-called isoenergetic, ‘balanced’ diets”. They also concluded that there is “probably little or no difference in weight loss and changes in cardiovascular risk factors up to two years of follow-up”.
South African media hailed the study as “debunking Banting”. Banting is the popular term for LCHF in South Africa. Others said that the “Noakes’ diet” was no healthier or better for dropping kilos than a balanced diet”. Reports also warned of the “dangers” of low-carb diets.
Click here to read: HOW LOW-FAT DIETS CAN KILL YOU
The Heart and Stroke Foundation of SA CEO said: “On the basis of current evidence, we cannot recommend a low-carbohydrate diet to the public.”
However, the Naudé review doesn’t mention Noakes or LCHF. It speaks generally of “some weight loss diets widely promoted through the media”. The review also mentions diets that “prescribe restriction of most vegetables and fruit” to achieve a “very low carbohydrate intake”.
It wasn’t possible for the authors to provide information or conclusions on low-carb diets for weight loss, Harcombe says, because they failed to study them.
More objective reporting of the Naudé review should have made that point, she says.
The average intake for 14 studies reviewed was 35% carbohydrate, 35% fat and 30% protein. That’s “substantially different” from the dietary composition of the low-carb diet that experts promote for therapeutic management of obesity and type 2 diabetes. Such a diet comprises 5% carbohydrate (less than 50g per day), moderate protein and high fat – up to 80%.
That’s the level at which patients achieve optimal benefits, say Noakes and Harcombe.
They have redone the meta-analysis without all the flaws. Their objective was to test whether the Naudé Review findings were robust. “We have demonstrated that they were not.”
Harcombe and Noakes have listed all the Naudé review errors that they found. Among these are that the researchers included studies that failed their own inclusion criteria. They also used invalid and subjective meta-analysis sub-grouping. As well, their data extraction was “repeatedly inaccurate”.
Of the 14 trials, four did not meet the author’s own criteria. One was a duplication of another included study. Thus, it shouldn’t have met anyone’s criteria. Therefore, the Naudé Review authors should only have analysed 10 studies. Most were lower in carbohydrate content than current public health dietary guidelines. However, they were not low-carb diets, says Harcombe.
The Naudé review’s main conclusion in the abstract was that in non-diabetic participants, their analysis showed “little or no difference in mean weight loss in the two groups at 3 – 6 months”. The authors based this on a study of 14 trials that were deemed “moderate-quality evidence”.
The objective of the Naudé review is also flawed, Harcombe and Noakes say. The authors aimed to compare the effects of “low (sic) carbohydrate and isoenergetic balanced weight loss diets in overweight and obese adults”. Isoenergetic simply means having the same calorie intake in both interventions.
Click here to read: HARCOMBE AIMS AT HEART OF HPCSA ‘TRIAL’
A key effect of the low-carb diet is to reduce hunger by increasing satiety despite a reduced energy intake, Harcombe says. Thus, subjects on the control diet in isoenergetic trials would have had to restrict their caloric intake voluntarily to match this effect. Thus, the authors negated the “uniquely satiating” advantage of low-carb diets.
These errors disadvantaged the lower-carb diets included in the systematic review.
For even more robustness in their re-examination, Harcombe and Noakes re-conducted the meta-analysis without the errors detected. They repeated it for the 10 studies that the authors’ own selection criteria would have included.
Their conclusion: using the review authors’ own criteria, the data confirmed that the lower-carb diet produced significantly greater weight loss than the balanced diet. In other words, the Naudé review shows the opposite of what the authors reported.
Click here to read: SCIENCE IN CRISIS: NOT JUST SUGAR SOURING FAITH IN EXPERTS
The SAMJ study has a limitation: Harcombe and Noakes looked at one part only of the Naudé review. However, they say the many errors in the analysis of the weight loss data made a material difference to the conclusions. The number of errors detected in that single section makes it “inconceivable that the remainder of the article is robust”.
Therefore, without the need to examine all sections of the article, the Naudé Review in its published form is “not robust and cannot be relied on”.
— Angharad P (@ape_owl78) December 2, 2016
If the authors had properly performed their meta-analysis, Harcombe says, they would have concluded that the lower-CHO diet produced greater weight loss than the “balanced” diet.
“This would have radically altered the nature of the message heard across SA after its publication,” she says. This might also have influenced “the eagerness of SA medical authorities to put the LCHF/Banting diet on public ‘trial’.”
Harcombe and Noakes are only relatively restrained in the implications of their re-examination. They say that the Naudé Review raises two reasonable questions:
- How could the published meta-analysis have included so many errors and the incorrect conclusion despite peer review?
- What is the chance that essentially all these errors favoured the “balanced diet” and disadvantaged the lower-carb diet? That’s especially when many of the review authors are on public record opposing lower- or low-carb diets and those who promote such eating plans?
They say that the Naudé Review researchers should “rectify the erroneous messages conveyed to the public” as a result of their many errors. Whether that will happen is anyone’s guess. My guess is it won’t because there’s probably too much at stake for the researchers and HPCSA.
Naudé declined to comment, other than to say that she and her co-authors will reply to the SAMJ.