By Marika Sboros
Did researchers at top South African universities deliberately make multiple mistakes in a major study? Was their aim to discredit low-carbohydrate, high-fat (LCHF) diets? And was their real target scientist Prof Tim Noakes?
Or are they just human, fallible and in this case, hopelessly error-prone?
A new study in the SAMJ (South African Medical Journal) is entitled The universities of Stellenbosch/Cape Town low-carbohydrate diet review: Mistake or mischief?. It re-examines a 2014 study by University of Cape Town (UCT) and Stellenbosch University (SU) scientists. That study is the “Naudé review” published in PLoS (Public Library of Science) One.
Lead author is Dr Celeste Naudé, head of Stellenbosch University (SU) Centre for Evidence-Based Health Care.
The authors of the new study have identified what they claim are many material errors that the scientists made and that terminally undermine their conclusions.
Many mistakes or mischief?
They raise the question: did the scientists make mistakes or mischief? Read on and make up your own mind.
Naudé’s co-authors include Prof Jimmy Volmink, head of SU Faculty of Healthy Sciences, and Prof Marjanne Senekal, head of UCT’s Human Nutrition Division.
Senekal became a consultant to the Health Professions Council of SA (HPCSA) in its ongoing trial of Prof Tim Noakes and the evidence for low-carb, high-healthy-fat (LCHF) eating.
She is in the same Health Sciences Faculty as Noakes at the University of Cape Town.
Noakes and British obesity researcher Dr Zoë Harcombe are the authors of a re-examination of the Naudé review in the SAMJ. They say that the “pivotal importance” of the review in the HPCSA’s trial against Noakes prompted them to re-examine it.
Harcombe was an expert witness for Noakes. At the October session of the trial, she presented evidence on the many flaws in the Naudé review. She and Noakes testified that the many errors have left the review fatally flawed.
The HPCSA has charged Noakes with unprofessional conduct for giving “unconventional advice to a breastfeeding mother on a social network”.
That was a single tweet in February 2014, saying that good first foods for infant weaning are LCHF. Johannesburg dietitian Claire Julsing Strydom reported him to the HPCSA.
Click here to read: The real beef dietitians have with Noakes.
Pillar of the case against him
Evidence at the fourth trial session in October 2016 shows that the HPCSA waited for the Naudé review before charging Noakes in September 2014. Evidence also shows that the HPCSA has based much of its case on the review.
The Naudé review authors concluded that low-carb diets awere“no more effective for producing weight loss than are high-carbohydrate, or so-called isoenergetic, ‘balanced’ diets”. They also concluded that there is “probably little or no difference in weight loss and changes in cardiovascular risk factors up to two years of follow-up”.
South African media hailed the study as “debunking Banting”. Banting is the popular term for LCHF in South Africa. Others said that the “Noakes’ diet” was no healthier or better for dropping kilos than a balanced diet”. Reports also warned of the “dangers” of low-carb diets.
Click here to read: How low-fat diets can kill you
The Heart and Stroke Foundation of SA CEO said: “On the basis of current evidence, we cannot recommend a low-carbohydrate diet to the public.”
However, the Naudé review doesn’t even mention Noakes, LCHF or Banting. It speaks generally of “some weight-loss diets widely promoted through the media”. The review also mentions diets that “prescribe restriction of most vegetables and fruit” to achieve a “very low carbohydrate intake”.
Litany of errors
Noakes and Harcombe have redone the meta-analysis to test whether the Naudé Review findings were robust. “We have demonstrated that they were not.”
Harcombe and Noakes have listed all the review errors that they found. Among these are that the researchers included studies that failed their own inclusion criteria. They also used invalid and subjective meta-analysis sub-grouping. As well, their data extraction was “repeatedly inaccurate”.
Of the 14 trials, four did not meet the author’s own criteria. One was a duplication of another included study. Thus, it shouldn’t have met anyone’s criteria. Therefore, the Naudé Review authors should only have analysed 10 studies. Most were lower in carbohydrate content than current public health dietary guidelines. However, they were not low-carb diets, said Harcombe.
The Naudé review’s main conclusion in the abstract was that in non-diabetic participants. Yet their analysis showed “little or no difference in mean weight loss in the two groups at 3 – 6 months”. The authors based this on a study of 14 trials that were deemed “moderate-quality evidence”.
The objective of the Naudé review was also flawed, Harcombe and Noakes said. The authors aimed to compare the effects of “low (sic) carbohydrate and isoenergetic balanced weight loss diets in overweight and obese adults”. Isoenergetic means having the same calorie intake in both interventions.
Aiming at heart of the nutrition matter
Click here to read: Harcombe aims at heart of HPCSA ‘trial’
A key effect of the low-carb diet was to reduce hunger by increasing satiety despite a reduced energy intake, Harcombe said. Thus, subjects on the control diet in isoenergetic trials would have had to restrict their caloric intake voluntarily to match this effect. Therefore, the authors negated the “uniquely satiating” advantage of low-carb diets.
These errors disadvantaged the lower-carb diets included in the systematic review, Noakes and Harcombe said.
For even more robustness in their re-examination, Harcombe and Noakes re-conducted the meta-analysis without the errors detected. They repeated it for the 10 studies that the authors’ own selection criteria would have included.
Their conclusion: using the review authors’ own criteria, the data confirmed that the lower-carb diet produced significantly greater weight loss than the balanced diet. In other words, the Naudé review shows the opposite of what the authors reported.
Click here to read: Science crisis: not just sugar souring faith in experts
The review byHarcombe and Noakes does have a limitation. They looked at one part only of the Naudé review. However, they said that the many errors in the analysis of the weight loss data made a material difference to the conclusions. The number of errors detected in that single section made it “inconceivable that the remainder of the article is robust”.
A failed study?
Therefore, without the need to examine all sections of the article, the Naudé Review in its published form was “not robust and cannot be relied on”.
I’m genuinely shocked. Naude et al’s manipulations are despicable. Post-truth science. https://t.co/jUSBHpDSSY @ProfTimNoakes @zoeharcombe
— Angharad P (@ape_owl78) December 2, 2016
If the authors had properly performed their meta-analysis, Harcombe said, they would have concluded that the lower-CHO diet produced greater weight loss than the “balanced” diet.
“This would have radically altered the nature of the message heard across SA after its publication,” she said. This might also have influenced “the eagerness of SA medical authorities to put the LCHF/Banting diet on public ‘trial’.”
Harcombe and Noakes say the Naudé Review researchers should “rectify the erroneous messages conveyed to the public” as a result of their many errors. Whether that will happen is anyone’s guess. My guess is it won’t because there’s probably too much at stake for the researchers and HPCSA.
Naudé declined to comment other than to say that she and her co-authors will reply to the SAMJ.
- Follow me on Twitter @MarikaSboros
- Like my Facebook Page
- Click here to subscribe for email notifications of Foodmed.net postings
Patients and the public often view personal anecdotes completely differently . Patients quite rightly want to know what works and have a connection with that person so it makes some sense. I cant speak for health professionals just myself. I view personal anecdotes as worthless particularly in this day and age where everyone wants to sell you their idea or product. Too many of them are not actually true.
One of the main problems with dietary studies in the first place is often that what someone reports they are doing is completely different to what they are actually doing. One of the key issues with LCHF is what constitutes low carb, how low is low and where does the evidence actually sit for say 40% carb or below 130g or below 50 or below 20g no one really knows yet.
The advantage of being a highly experienced dietitian is having consulted with thousands of individuals on all manner of diets and seeing how there has never been a one size fits all and never will be. We focus too much on macro nutrients. I have consulted with people with and without diabetes who have lost weight and regained good control on all manner of diets from 60% carb in vegetarians to virtually nil with some LCHF followers. The carb content had nothing to do with it their dedication to change did.
The other key aspect is what is actually more important: weight, overall health, bowel health, mental health,cancer, longevity? Given that we are living longer, we don’t actually die from obesity of diabetes. In most cases its other causes . currently there is some good information coming out on LCHF but we need to learn more and it’s an acceptable approach. Might it have short- and long-term problems? Yes. All dietary extremes have in the past. I note that many LCHF followers have become very nasty and vindictive on sites and towards my profession and that is very concerning.
“I view personal anecdotes as worthless”
You probably view outcomes as worthless too.
“Man up and wait for your amputation, dialysis and retinopathy”
Dietary extremes? I wonder what would be classed as the middle ground of diets. Personally I find that a low-carb diet would be that middle ground as high-carb would be potentially the exact opposite of no carb? We know that the human body requires carbohydrate, however it seems to me that carbohydrate from a carbohydrate food source is not required as triglycerides, when broken down, supply us with glucose and as I found out from my own personal experience, protein that is not used also is used as carbohydrate as this is also changed into sugars. It’s fascinating for me that it seems that the human body can only store fat if it has been changed into sugars. Surely this raises the question, why bother with eating any carbs at all? And if we do why wouldn’t we expect to get fat….
Right on, Steven. I’m always amused when doctors or dietitians call low-carb diets “extreme. In nature, there is no such thing as what they call a “balanced” diet. Low-carb isn’t a fad or extreme way of eating. It is a perfectly natural way of eating. High-carb, processed, denatured foods are not natural.
Isn’t it amazing that the same dieticians who praise veg(etari)ans have a hissy fit the moment someone admits to reducing carbs or eliminating wheat on the grounds that it is “dangerous” to eliminate an entire food group? Eliminating meat is a good thing? How about eliminating trans fats, or arsenic?
Sorry Catherine Collins RD found it necessary to threaten you with legal action following one of my posts.
Here’s the thread where Chris (Cashin) a dietician also posts as “Ally” and is supported in her semiliterate rants by Collins
http://www.drbriffa.com/2007/10/03/why-is-the-dietary-advice-given-to-diabetics-so-often-so-woefully-inadequate/
several other dieticians join in the feeding frenzy
http://www.drbriffa.com/2008/06/25/why-might-a-leading-diabetes-charity-offer-dietary-advice-that-is-likely-to-increase-the-need-for-medication/#comments
is where they reattacked him, so the history of attacking doctors for “inappropriate” reversal of diabetes has a long history.
I wonder if Briffa himself has archived the now defunct blog where they threatened to have his licence?
Meanwhile what are dieticians doing today, nearly a decade later while diabetes numbers have, what, doubled?
http://lowcarbdiabetic.forumotion.co.uk/t2743-type-2-diabetic-320-reasons-to-avoid-dietitians
Still bleating about having no evidence, if they’d started ten years ago they would have ten years of “evidence” by now. BTW the study Cashin refers to which she told us on a forum would “prove once and for all” the dangers of low carb diets never got published, I wonder why that was?
Hi. I’ve added a comment to the Naudé review on PubMed so that people can be alerted to the paper in the SAMJ: https://www.ncbi.nlm.nih.gov/pubmed/25007189/#comments
Since when is anecdotal evidence “unproven”?
Here’s just one example of “proven” evidence: 72-year old male, dropped 34kg, no longer takes any medicine for high blood pressure, cholesterol or Diabetes 2, plays veteran tennis at provincial level, holds down full time job in stressful environment.
Big Pharma and Big Food are crying foul at every opportunity but they are losing the battle to LCHF.
I agree. Personal anecdote is the only thing that I listen to with real interest in this debate. The number of success stories (anecdotes) is huge and growing. It is an information goldmine that gives mainstream “researchers” sleepless nights. If you were to take the totality of anecdotal evidence and weigh it up against the proof that Nick insists on then anecdote would win the day every single time. This might not suit those steeped in outmoded dogma, but who cares.
At age 69, I am also one of these success stories as I am no longer obese, hypertensive, pre diabetic or dyslipidemic BECAUSE I switched to a low carb lifestyle without any help from a doctor or a dietitian. I have bought into the wisdom of crowds because it is the most effective source of real information. This is all about becoming informed and people like me are quite capable of being my own physician in this specific context. I do not need a dietitian to tell me the obvious. Furthermore I do not allow the so called experts to confuse me with their own personal interpretation of current trends. Most of these experts are ill informed
Precisely. As I just wrote in another thread, what is dismissed as “just another anecdote” is actually a carefully controlled scientific experiment with the SAME subject pre and post intervention, pretty much Gold Standard and requiring no statistical shenanigans.
All ill people and an increasing number of well people are routinely monitored for things like weight, blood glucose/HbA1c (insulin would be a major wish that will never be fulfilled), blood pressure, lipid panels, sometimes CRP, homocysteine and other more obscure markers.
Almost without exception, everyone on a low carb diet will have already tried and failed on low fat.
Dieticians and many doctors (there’s a small but growing band who think differently) call this “anecdotes” and a carefully crafted study of 30 subjects bought and paid for by the Sugar Bureau and published by Diabetes UK “Evidence”.
Thus you end up with doctors like mine – who is far from clueless – remarking that “by now we would have expected you to be on two or three diabetes drugs” without showing much interest in how I achieved this.
Our Dr David Unwin of Southport claims to be saving our NHS up to £45 000/year using low carb diets. He or someone else has calculated that if all UK GPs followed suit, the NHS would save nearly half a BILLION pounds per year.
Of course another way of looking at this is that drug companies would LOSE half a billion pounds per year, which is why doctors are not told how to achieve this.
Well done, guys. Three personal, incorrect insults and Steve, with the typically unproven anecdote . WThat will change the minds of highly analytical individuals who have studied nutrition for five years then spoken to thousands of patients on hundreds of diets. Steve, it’s good you have achieved what you have on your LCHF diet . Re replication of your results, well actually I’m three years older than you my body fat percentage is similar and practically every male dietitian I know and most atheletes are similar. I am not sure why it would be so importamt to have the body fat percentage you have mentioned. It’s not associated with any positive outcomes . Do we just eat food to have our desired body shape ?
Hi Nick. I mentioned my body fat percentage to highlight what happens as a consequence of a LCHF diet. It wasn’t a goal as such, although very welcome. And you are quite right in suggesting that many athletes and dietitians will be similar but l am none of these and as such l feel that this is of benefit to all. All the benefits with only the effort of eating less carbohydrate and more fat with no extra exercise. That seems pretty good to me.
As a LCHF follower for nearly a decade. I have seen and felt personally the great benefits of following such eating habits. I wasn’t obese or diabetic but still my blood pressure lowered. I felt more relaxed and had less anxiety. I was able to keep working and playing for longer, whilst eating only when my body required it. Currently, I am around 10 % body fat and at 43 I am in the greatest shape of my life. If this isn’t a healthy way of eating and living show me a high-carb lover the same. You probably can’t…
Have you seen the Outside Magazine article on Prof Noakes? Poorly written is my
assessment. When he has Noakes quoted as saying he changed his mind from “…high-carb to low-fat.” you know the author has completely fumbled the issue.
Sigh. Yes, many journalists who should know better have written rubbish about Prof Noakes. Raises the question why they haven’t bothered to do proper research.
Come on people you have got to be kidding. I am a dietitian and have heard Prof Noakes speak twice after his evangalstic change to LCHF. I walked out on the second one. I was tired of having a loud opinionated bully yell at me and the audience telling us we are killing people. This guy’s in the perfect position to produce more high-quality evidence too back his point but instead he brings up poor studies and anecdotes . We would love it if he could bring more quality evidence to the table. The fact that he doesn’t is because it’s not there and no amount of yelling will improve that. I don’t feel threatened, just disappointed that someone in his position doesn’t actually help the conversation. No dietitian on planet earth would or will advise a diet that is detrimental. Knock yourself out if you want to go LCHF, but stop pretending your evidence is sound. Get better evidence and we will change tomorrow.
Problem with dietitians is they are too proud to admit they are wrong despite seeing how millions of people have benefited from LCHF diet. Prof Noakes is a hero, not many dietitians will ever help as many people as he did. Sour grapes maybe.
O come on Nick!! Do you still cling to the flat earth theory?!! Qualifying as a dietician means nothing if your science is outdated. Get off your high horse and study low carb factual data. As usual South Africa is way behind with new science.
But. You. ARE. Advising. A Diet. That. Is. Detrimental.
I suppose that it is always difficult to consider the possibility that everything you believe may be wrong. Worse still, that this may have adversely affected the health of hundreds of people. Ponder that. Noakes did and then he changed his mind. The fact that you walked out and refused to listen to an opposing viewpoint tends to suggest that you are simply not open to new information. Do you make this clear to your patients when consulting?
By the way, there are times when I find the intolerance on both sides of the debate (including the Banters) a complete pain the neck and your comment about bullying has some validity. There is nothing about any body of scientific information that is cast in concrete and the low carb crowd are starting to sound a bit too convinced of their own rectitude. They should beware the dangers of overreaction to previous poor global health management. There is nothing worse than using knowledge as a battering ram.
Agree with all you say, Anton. There is much collateral damage going on in this debate, including to dietitians’ reputations. Many are fine, intelligent people, hugely endowed with integrity. I’m meeting more who don’t have closed minds and who don’t think that their associations are speaking well for them. The debate has become extraordinarily divisive. I hope that will change sooner rather than later. One problem with reflecting both sides of this debate is that the “other” – orthodox – side is usually reluctant to speak on platforms that reflect views with which they disagree. That’s not helpful. They need to open up and take criticism on the chin, not personally.
I think they are very aware that if they speak out their jobs will be on the line. Certainly I’ve heard of not a few clueful doctors here (UK) who have been warned that they Must Not encourage, or even discuss, LCHF, or prescribe glucose test strips to diabetics not on insulin. They whisper this quietly to their patients hoping Big Brother is not listening.
Debunking the evidence that is being brought against you in a trial is not only fair but required. You will see the exact same thing in both civil and criminal trials, and it is just as important here. As soon as that paper was introduced as evidence that his approach was unsound, it needed to be examined.
I find a couple of your statements interesting, however:
“No dietitian on planet earth would or will advise a diet that is detrimental.” –
While they may not knowingly advise a diet that it detrimental, if the assumptions on which they base their conclusions are wrong, their conclusions (while perfectly logical and sound) will be wrong. No one is suggesting that all dietitions are doing it maliciously, but in at least some cases it is very difficult to ascribe other motives to it… Joslins, the Harvard-based diabetes center, suggests 40% or more of calories from carbs for diabetics. There is no way that you can tell me that that is not a detrimental dietary regime for diabetics, and yet there it is… And there are lots more examples…
“Knock yourself out if you want to go LCHF, but stop pretending your evidence is sound. Get better evidence and we will change tomorrow.” – There is tons of really good evidence, including not just epidemiological studies but proper clinical trials, like Christopher Gardner’s A to Z study out of Stanford, for example. Other people shouldn’t have to do your professional research for you, however.