By Marika Sboros
Mistake or mischief? Did top scientists at Stellenbosch and Cape Town universities honestly make so many mistakes in a major study? Did they really not know the Health Professions Council of SA (HPCSA) would use it to charge scientist Prof Tim Noakes? Or was there something a little more contrived behind their research?
British obesity researcher Dr Zoë Harcombe asked those questions in her evidence-in-chief on day six of the HPCSA’s hearing against Noakes in Cape Town today. Harcombe is one of three expert witnesses for Noakes who have flown in from the UK, US and New Zealand.
The public has dubbed them “Tim’s Angels”. There was nothing angelic about Harcombe’s takedown of the study known as the “Naudé Review”. Here’s what Harcombe had to say about it:
The research in question is a systematic review and meta-analysis of 19 studies. The PLoS (Public Library of Science) One journal published it in 2014.
Click here to read the full text of Low Carbohydrate versus Isoenergetic Balanced Diets for Reducing Weight and Cardiovascular Risk: A Systematic Review and Meta-Analysis.
It is also known as the “Stellenbosch Review”. It’s probably more correct to call it the Stellenbosch, University of Cape Town and Cochrane Collaboration Review.
Lead author is Dr Celeste Naudé of Stellenbosch University’s Centre for Evidence-based Healthcare.
Co-authors of the study include Dr Marjanne Senekal. She is associate professor and head of UCT Division of Human Nutrition.Senekal is a consultant to the HPCSA in the case against Noakes.
Another author is Prof Jimmy Volmink, dean of Stellenbosch Faculty of Health Sciences and a member of the Cochrane Collaboration Centre at the Medical Research Council in South Africa.
Noakes faces a charge unprofessional conduct for giving unconventional advice to a breastfeeding mother on a social network (Twitter). That was a single tweet in February 2014 saying good first foods for infant weaning are low-carb, high-fat (LCHF).
Johannesburg dietitian Claire Julsing Strydom just happened to be on Twitter at the same time and reported him to the HPCSA. Strydom was president of the Association for Dietetics in SA (ADSA) at the time. For background click here to read The real beef dietitians have with him.
Naudé and her co-authors have concluded that “when people eat a similar amount of energy in the low-carb and balanced diets, there was no difference in weight-loss after three to six months.” In other words, low-carb diets are no better for weight loss than conventional, high-carb, low-fat “balanced diets”.
Harcombe is an expert in systematic review and meta-analysis. She used a formidable forensic scalpel to dissect the Naudé Review and expose all the errors. Many are significant. All but one favour the control, “balanced diet”.
Harcombe said the researchers could just as easily have concluded the opposite: a ‘balanced” (high-carb) diet is no better than a low-carb diet for producing weight loss.
Among the errors, Harcombe said the researchers have:
- Included studies that failed their own inclusion criteria.
- Used invalid and subjective meta-analysis sub-grouping.
- Made data extraction that was “repeatedly inaccurate”. One instance of data extraction was so erroneous, Harcombe called it “absurd”, another inexplicable”.
A major limitation of the review, Harcombe said, is that the authors claim to have reviewed evidence for low-carb diet but have not done so. In other words, they “could not judge low-carbohydrate diets because they did not study them”.
Click here to read: ARE CARDIOLOGISTS AT HEART OF THIS TRIAL?
The average dietary intake for 14 studies was 35% carbohydrate, 35% fat and 30% protein. That’s very different from the 5% to 10% carbohydrate and 80% to 85% fat of a genuinely low-carb, high-fat diet.
Harcombe said the review also set isocaloric (having similar caloric values) as a criterion. This negated the satiety advantage of low-carb diets. Satiety is a key effect of the low-carb diet despite a reduced energy intake. Thus, researchers in isoenergetic trials would have had to restrict the caloric intake voluntarily of subjects on the control diet to match this effect.
If the researchers were to redo their research without all the errors, Harcombe said, they would have to make a very different conclusion. They would have to find that the low-carb diet actually works better than the (low-fat, high-carb) control diet for weight loss.
— Marika Sboros (@MarikaSboros) October 24, 2016
Harcombe and Noakes have done just that in a re-analysis of the Naudé Review detailing all the errors. Their study has gone through the peer-review process and a journal has accepted it for publication.
They have only re-examined one part of the Naudé Review. However, given so many errors in one section, they believe it is unlikely that the rest of the paper is robust.
In her evidence, Harcombe described as “interesting” the media reporting of the review that ensued. Incriminating might be a better adjective.
The authors never mention LCHF, Banting or Noakes. Yet media reports quickly claimed that the review “debunked the Banting diet”. Other reports personalised the message and referred to Noakes as a “celebrity professor”. One said the Naudé Review proved that “Noakes’ low-carb diet is not healthier”.
One report quoted chief executive of the Heart and Stroke Foundation of South Africa (HSFSA) Dr Vash Mungal-Singh saying: “The current evidence means we cannot recommend a low-carbohydrate diet to the public.”
If Naudé and her co-authors did not proactively relay those messages, it is “interesting” that they did nothing to correct them at the time – or have done since.
Harcombe said these media reports, apart from being incorrect, had “potential for harm”. The many people who are on “Banting”-style diets to treat obesity, diabetes and other conditions and benefitting may have felt worried enough to stop.
She and Noakes uncovered other problems, including that some studies lacked generalisability to whole populations. Two, for example, looked at males only. Overall, Harcombe described the review as “unprofessional” – and worse: “fraud”.
Therefore, I’d say a call for the journal to retract the study cannot be far away.
It is littered with so many errors it raises other disturbing questions. Like how did the authors manage to publish their study at all.
In the background to the study, the authors refer rather archly to “some weight-loss diets widely promoted through the media”. However, the only one they name is the Atkins diet. They say “these diets” recommend a regimen “greatly restricting carbohydrates, with increased protein and unrestricted total and saturated fat intake”. Most of the low-carb diets I’ve ever heard of don’t advise unrestricted total and saturated fat intake. Most put a percentage onto it.
The researchers say nary a word about Noakes or Banting.
Given the controversy and fury swirling around both in medical and dietetic circles at the time, that oversight can seem far more by design than default. The researchers say it is “plausible that these low-carbohydrates could be harmful, especially over the longer term”. They give references, but that’s a statement loaded with bias if ever I read one.
The Naudé Review may also signal a deep rot in the peer-review process. In an article in the Journal of the Royal Society of Medicine in 2006, former BMJ editor Dr Richard Smith, CE of United Health Europe, is damning about the process.
He says there is “little evidence on the effectiveness of peer review, but we have considerable evidence on its defects.
“In addition to being poor at detecting gross defects and almost useless for detecting fraud it is slow, expensive, profligate of academic time, highly subjective, something of a lottery, prone to bias, and easily abused.”
If the Naude Review really does turn out to be more mischief than mistake – as is looking very likely – it will be yet another example of that abuse. It will also be another lesson in unintended consequences that are piling up for the HPCSA in prosecuting Noakes.
- Click here for full, exclusive coverage of The Noakes Trial
- Follow me on Twitter @MarikaSboros
- Like my Facebook Page
- Subscribe for email notifications of Foodmed.net postings