By Marika Sboros
Mistake or mischief? Did top scientists at Stellenbosch and Cape Town universities really make so many honest mistakes in their study? Did they genuinely not know that the Health Professions Council of SA (HPCSA) would use it to charge scientist Prof Tim Noakes?
Or was there something more contrived, even sinister in intent behind their research?
British obesity researcher Dr Zoë Harcombe asked those questions and more in her evidence-in-chief on day six of the HPCSA’s hearing against Noakes in Cape Town today. Harcombe is one of three expert witnesses for Noakes who have flown in from the UK, US and New Zealand.
The public has dubbed them “Tim’s Angels”. There was nothing angelic about Harcombe’s takedown of the study known as the “Naudé Review”.
Here’s what Harcombe said about it:
Pinpointing the errors
The research is a systematic review and meta-analysis of 19 studies. The PLoS (Public Library of Science) One journal published it in 2014.
Click here to read the full text: Low Carbohydrate versus Isoenergetic Balanced Diets for Reducing Weight and Cardiovascular Risk: A Systematic Review and Meta-Analysis.
It is also known as the “Stellenbosch Review”.
Lead author is Dr Celeste Naudé of Stellenbosch University’s Centre for Evidence-based Healthcare. Co-authors include Dr Marjanne Senekal. She is associate professor and head of UCT Division of Human Nutrition. Despite being a colleague of Noakes in the same faculty at UCT Senekal has become a consultant to the HPCSA against him.
Another author is Prof Jimmy Volmink, dean of Stellenbosch Faculty of Health Sciences and a member of the Cochrane Collaboration Centre at the Medical Research Council in South Africa.
Noakes faces a charge unprofessional conduct for giving unconventional advice to a breastfeeding mother on a social network (Twitter). That was a single tweet in February 2014. He tweeted that good first foods for infant weaning are low-carb, high-fat (LCHF).
A ‘horrified’ dietitian
Johannesburg dietitian Claire Julsing Strydom just happened to be on Twitter at the same time. She tweeted in reply that she was “horrified” at its contents. She reported him to the HPCSA the next day.
Strydom was president of the Association for Dietetics in SA (ADSA) at the time. For background click here to read: The real beef dietitians have with him.
Naudé and her co-authors have concluded that “when people eat a similar amount of energy in the low-carb and balanced diets, there was no difference in weight-loss after three to six months.” In other words, low-carb diets are no better for weight loss than conventional, high-carb, low-fat “balanced diets”.
Harcombe is an expert in systematic review and meta-analysis. She used a formidable forensic scalpel to dissect the Naudé Review and expose what she claimed were significant errors. All but one favour the control, “balanced diet”.
Among the errors, Harcombe said that the researchers had:
- Included studies that failed their own inclusion criteria.
- Used invalid and subjective meta-analysis sub-grouping.
- Made data extraction that was “repeatedly inaccurate”. One instance of data extraction was so erroneous, Harcombe called it “absurd”, another inexplicable”.
More terminal study errors
A major limitation of the review, Harcombe said, was that the authors claimed to have reviewed evidence for low-carb diet but had not done so. In other words, they “could not judge low-carbohydrate diets because they did not study them”.
Click here to read: Are cardiologists at the heart of this trial?
The average dietary intake for 14 studies was 35% carbohydrate, 35% fat and 30% protein, Harcombe said. That was very different from the 5% to 10% carbohydrate and 80% to 85% fat of a genuinely low-carb, high-fat diet, she said.
Harcombe said that the review also set isocaloric (having similar caloric values) as a criterion. This negated the satiety advantage of low-carb diets. Satiety is a key effect of the low-carb diet despite a reduced energy intake, she said. Thus, researchers in isoenergetic trials would have had to restrict the caloric intake voluntarily of subjects on the control diet to match this effect.
If the researchers were to redo their research without all the errors, Harcombe said, they would have had to come to a very different conclusion. They would have to find that the low-carb diet actually worked better than the (low-fat, high-carb) control diet for weight loss.
— Marika Sboros (@MarikaSboros) October 24, 201
Harcombe and Noakes have done just that in a re-analysis of the Naudé Review detailing all the errors. Their study has gone through the peer-review process and a journal has accepted it for publication.
They have only re-examined one part of the Naudé Review. However, given so many errors in one section, they believe it is unlikely that the rest of the paper is robust.
LCHF in media headlines
In her evidence, Harcombe described as “interesting” the media reporting of the review that ensued. (Incriminating might be a better adjective.)
The authors never mentioned LCHF, Banting or Noakes. Yet media reports quickly claimed that the review “debunked the Banting diet”, as she showed. Other reports personalised the message and referred to Noakes as a “celebrity professor”. One said that the Naudé Review proved that “Noakes’ low-carb diet is not healthier”.
One report quoted chief executive of the Heart and Stroke Foundation of South Africa (HSFSA) Dr Vash Mungal-Singh saying that “the current evidence means we cannot recommend a low-carbohydrate diet to the public.”
If Naudé and her co-authors did not proactively relay those messages, it was “interesting” that they did nothing to correct them at the time – or have done since.
Harcombe said that these media reports, apart from being incorrect, had “potential for harm”. The many people on “Banting”-style diets to treat obesity, diabetes and other conditions and benefitting might have felt worried enough to stop.
Harcombe and Noakes uncovered other problems, including that some studies lacked generalisability to whole populations, she said. Two, for example, looked at males only. Overall, Harcombe described the review as “unprofessional” at best, at worst, possible “fraud”.
Therefore, I’d say that a call for the journal to retract the study cannot be far away.
Rot in the peer review process
The Naudé Review may also signal a deep rot in the peer-review process, as other researchers have suggested. In an article in the Journal of the Royal Society of Medicine in 2006, former BMJ editor Dr Richard Smith, CE of United Health Europe, was damning about the process.
He said that there was “little evidence on the effectiveness of peer review, but we have considerable evidence on its defects.
“In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.”
If the Naude Review really does turn out to be more mischief than mistake – as is looking likely – it will be yet another example of that abuse, as Harcombe indicated in her evidence. Legal experts say that it will also be another lesson in unintended consequences piling up for the HPCSA and all the dietitians involved in prosecuting Noakes.
- Click here for full, exclusive coverage of The Noakes Trial
- Follow me on Twitter @MarikaSboros
- Like my Facebook Page
- Subscribe for email notifications of Foodmed.net postings