By Marika Sboros
Did researchers at top South African universities deliberately make multiple mistakes in a major study? Was their aim to discredit low-carbohydrate, high-fat (LCHF) diets? And was their real target scientist Prof Tim Noakes?
Or are they just human, fallible and in this case, hopelessly error-prone?
A new study in the SAMJ (South African Medical Journal) is entitled The universities of Stellenbosch/Cape Town low-carbohydrate diet review: Mistake or mischief?. It re-examines a 2014 study by University of Cape Town (UCT) and Stellenbosch University (SU) scientists. That study is the “Naudé review” published in PLoS (Public Library of Science) One.
Lead author is Dr Celeste Naudé, head of Stellenbosch University (SU) Centre for Evidence-Based Health Care.
The authors of the new study have identified what they claim are many material errors that the scientists made and that terminally undermine their conclusions.
Many mistakes or mischief?
They raise the question: did the scientists make mistakes or mischief? Read on and make up your own mind.
Naudé’s co-authors include Prof Jimmy Volmink, head of SU Faculty of Healthy Sciences, and Prof Marjanne Senekal, head of UCT’s Human Nutrition Division.
Senekal became a consultant to the Health Professions Council of SA (HPCSA) in its ongoing trial of Prof Tim Noakes and the evidence for low-carb, high-healthy-fat (LCHF) eating.
She is in the same Health Sciences Faculty as Noakes at the University of Cape Town.
Noakes and British obesity researcher Dr Zoë Harcombe are the authors of a re-examination of the Naudé review in the SAMJ. They say that the “pivotal importance” of the review in the HPCSA’s trial against Noakes prompted them to re-examine it.
Harcombe was an expert witness for Noakes. At the October session of the trial, she presented evidence on the many flaws in the Naudé review. She and Noakes testified that the many errors have left the review fatally flawed.
The HPCSA has charged Noakes with unprofessional conduct for giving “unconventional advice to a breastfeeding mother on a social network”.
That was a single tweet in February 2014, saying that good first foods for infant weaning are LCHF. Johannesburg dietitian Claire Julsing Strydom reported him to the HPCSA.
Click here to read: The real beef dietitians have with Noakes.
Pillar of the case against him
Evidence at the fourth trial session in October 2016 shows that the HPCSA waited for the Naudé review before charging Noakes in September 2014. Evidence also shows that the HPCSA has based much of its case on the review.
The Naudé review authors concluded that low-carb diets awere“no more effective for producing weight loss than are high-carbohydrate, or so-called isoenergetic, ‘balanced’ diets”. They also concluded that there is “probably little or no difference in weight loss and changes in cardiovascular risk factors up to two years of follow-up”.
South African media hailed the study as “debunking Banting”. Banting is the popular term for LCHF in South Africa. Others said that the “Noakes’ diet” was no healthier or better for dropping kilos than a balanced diet”. Reports also warned of the “dangers” of low-carb diets.
Click here to read: How low-fat diets can kill you
The Heart and Stroke Foundation of SA CEO said: “On the basis of current evidence, we cannot recommend a low-carbohydrate diet to the public.”
However, the Naudé review doesn’t even mention Noakes, LCHF or Banting. It speaks generally of “some weight-loss diets widely promoted through the media”. The review also mentions diets that “prescribe restriction of most vegetables and fruit” to achieve a “very low carbohydrate intake”.
Litany of errors
Noakes and Harcombe have redone the meta-analysis to test whether the Naudé Review findings were robust. “We have demonstrated that they were not.”
Harcombe and Noakes have listed all the review errors that they found. Among these are that the researchers included studies that failed their own inclusion criteria. They also used invalid and subjective meta-analysis sub-grouping. As well, their data extraction was “repeatedly inaccurate”.
Of the 14 trials, four did not meet the author’s own criteria. One was a duplication of another included study. Thus, it shouldn’t have met anyone’s criteria. Therefore, the Naudé Review authors should only have analysed 10 studies. Most were lower in carbohydrate content than current public health dietary guidelines. However, they were not low-carb diets, said Harcombe.
The Naudé review’s main conclusion in the abstract was that in non-diabetic participants. Yet their analysis showed “little or no difference in mean weight loss in the two groups at 3 – 6 months”. The authors based this on a study of 14 trials that were deemed “moderate-quality evidence”.
The objective of the Naudé review was also flawed, Harcombe and Noakes said. The authors aimed to compare the effects of “low (sic) carbohydrate and isoenergetic balanced weight loss diets in overweight and obese adults”. Isoenergetic means having the same calorie intake in both interventions.
Aiming at heart of the nutrition matter
Click here to read: Harcombe aims at heart of HPCSA ‘trial’
A key effect of the low-carb diet was to reduce hunger by increasing satiety despite a reduced energy intake, Harcombe said. Thus, subjects on the control diet in isoenergetic trials would have had to restrict their caloric intake voluntarily to match this effect. Therefore, the authors negated the “uniquely satiating” advantage of low-carb diets.
These errors disadvantaged the lower-carb diets included in the systematic review, Noakes and Harcombe said.
For even more robustness in their re-examination, Harcombe and Noakes re-conducted the meta-analysis without the errors detected. They repeated it for the 10 studies that the authors’ own selection criteria would have included.
Their conclusion: using the review authors’ own criteria, the data confirmed that the lower-carb diet produced significantly greater weight loss than the balanced diet. In other words, the Naudé review shows the opposite of what the authors reported.
Click here to read: Science crisis: not just sugar souring faith in experts
The review byHarcombe and Noakes does have a limitation. They looked at one part only of the Naudé review. However, they said that the many errors in the analysis of the weight loss data made a material difference to the conclusions. The number of errors detected in that single section made it “inconceivable that the remainder of the article is robust”.
A failed study?
Therefore, without the need to examine all sections of the article, the Naudé Review in its published form was “not robust and cannot be relied on”.
— Angharad P (@ape_owl78) December 2, 2016
If the authors had properly performed their meta-analysis, Harcombe said, they would have concluded that the lower-CHO diet produced greater weight loss than the “balanced” diet.
“This would have radically altered the nature of the message heard across SA after its publication,” she said. This might also have influenced “the eagerness of SA medical authorities to put the LCHF/Banting diet on public ‘trial’.”
Harcombe and Noakes say the Naudé Review researchers should “rectify the erroneous messages conveyed to the public” as a result of their many errors. Whether that will happen is anyone’s guess. My guess is it won’t because there’s probably too much at stake for the researchers and HPCSA.
Naudé declined to comment other than to say that she and her co-authors will reply to the SAMJ.
- Follow me on Twitter @MarikaSboros
- Like my Facebook Page
- Click here to subscribe for email notifications of Foodmed.net postings