Doctors and dietitians love to say the advice they dish out on diet is ‘evidence-based’. The evidence for that leaves a very bad taste in some researchers’ mouths. They say that in its current state, evidence-based medicine props up official dietary guidelines. These guidelines have contributed to the unnecessary suffering and premature deaths of millions worldwide. They say that in nutrition, evidence-based medicine (EBM) has done an excellent job – mostly of improving some scientists, doctors, dietitians and pharmaceutical and food companies’ bottom lines.
US biochemistry professor Dr Richard Feinman is a vociferous critic of the current state of EBM and the influential US dietary guidelines that draw on it. Feinman is professor at SUNY (State University of New York) Medical School’s department of cell biology. He agrees that EBM is little more than ‘the position of experts on one lucrative side of a scientific conflict’. Here, he dissects EBM with surgical, scientific precision, to expose its inability to save lives from the growing pandemics of obesity and diabetes. – Marika Sboros.
By Richard Feinman*
According to the Journal of the American Medical Association (JAMA), the principle of “evidence-based medicine” (EBM) arose in the 1990s. Researchers invoke it widely in the medical literature as a kind of certification that the conclusions of the authors are not mere opinions, but have compelling information in biomedical science backing them up.
It sounds good. Or does it?
It is certainly self-serving and a little suspicious, somewhat like Nixon assuring us that he was not a crook.
Evidence-based medicine? What were we doing before? How was Pasteur able to function in the absence of such an idea?
One thing to think about is that evidence is what is introduced into courts of law. But not all evidence is admissible. A judge decides what is admissible.There are many precedents, in particular, on what constitutes scientific evidence in a legal proceeding.
Evidence-based medicine relies on a hierarchy of levels of evidence, with the random controlled trial (RCT) as the highest and expert opinion as the lowest. (See Table 1 below). Recommendations from health agencies and awarding of research grants are frequently justified on conformity to EBM or at least on their placing primary importance on RCTs.
Early in 2011, the US Department of Agriculture (USDA) released its 2010 Dietary Guidelines for Americans. With the dates suggesting the backward-looking nature of the guidelines, they were nonetheless based on the 2010 report of a prestigious committee, the Dietary Guidelines Advisory Committee (DGAC).
The DGAC, in turn, made much of their reliance on a new Nutrition Evidence Library (NEL). I and my colleagues were invited to submit a critique of the report by the journal, Nutrition.
The editor, Michael Meguid indicated that the journal wanted a balanced report – pros and cons.
I called Dr Meguid:
RDF: You know, the report is not particularly balanced. I’m not sure how you write a balanced review of an unbalanced report.
MM: You can make the critique as strong as you like as long as you carefully document everything. But what’s your main problem with the report?
RDF: Well, it makes very strong recommendations in the face of contradictory evidence.
MM: Make that the title of your article.
So we wrote an article, In the face of contradictory evidence: Report of the Dietary Guidelines for Americans Committee (DGAC). The journal was kind enough to make it an open access article, available here. In the end, we were one-upped by Steven Malanga, whose article in the New York Post was called Fed’s Food Fog.
For sure, both the report and the final guidelines were the proverbial camel-like production of a committee, tedious, repetitive and stylistic dreadful. But what about the NEL?
What about the evidence? Style aside, wasn’t this evidence-based medicine? Where do these guidelines come from?
The assumption is that evidence follows its etymologic roots, stuff that is visible, stuff that comes from the sensible and true avouch of our own eyes. In fact, it is most often applied, as in the case of the DGAC, to the most controversial and contentious subjects. Calling something evidence is not enough. So what happens in courts of law?
In a court of law, a judge decides on whether the jury can hear the evidence. Who decides admissibility of the evidence in EBM?
Science is a human activity. Conflict, controversy and a resistance to new ideas are well-known. That’s even in the so-called hard, more mathematical sciences. And even where there are no outside forces as in the case of Galileo.
In the 20th century, conflicts do not generally impede progress for long. Especially in the physical sciences, there is usually agreement on basic assumptions and on the rules of logic, allowing ultimate acceptance of strong evidence. Competing theories may coexist. Supporters of both are likely to admit that they are awaiting reconciliation.
What happens when the spontaneous process of conflict resolution in science breaks down? What happens in conditions where scientific disagreement is strong and a majority position becomes so dominant that it controls the funding and publication of scientific work and can ignore or repress contradictory evidence and repress exposition of alternative theories.
In essence, how do we deal with a recapitulation of the case of Galileo?
There is no system to decide on the admissibility in the cases considered by EBM.
I am not the first to point out that EBM is largely the position of experts on one side of a scientific conflict, the lowest level of evidence on traditional evidence-based medicine scales (eg “Level III: Opinions of respected authorities… of the US Preventive Services Task Force Systems” in Table 1).
In this situation where science cannot police itself, we have to look for some outside guidance. What do the courts do? As one would expect, there is a long and extensive history of the legal system’s attempt to deal with what constitutes scientific evidence. On the chance that the legal perspective may help, I will discuss some of the issues.
A key decision in the history of science in the courts is Frye v United States. In 1923, a Federal Appeals court ruled that the opinions of experts have to be supported by a scientific community. Frye had been convicted of second-degree murder but appealed on the grounds that he had successfully passed a lie-detector test.
At that time, the device was a simple blood pressure machine and an expert witness testified as to the results.
The court ruled that the lie-detector test “has not yet gained such standing and scientific recognition among physiological and psychological authorities as would justify the courts in admitting expert testimony”.
The court affirmed the judgment of the lower court.
The ruling in Frye gave rise to the idea of “general acceptance”. By analogy, this appears to be the main principle in the admissibility of evidence in the nutrition world.
Sufficiently well established that it could be included in a biochemistry text is the idea that “consumption of saturated fats is positively associated with high levels of total plasma cholesterol and LDL cholesterol and an increased risk of CHD”.
Known to students as “the Lippincott Book” by Harvey and Ferrier, it is the best-selling biochemistry book in the world. It is correct when it states that “most experts strongly advise limiting intake of saturated fats”.
Most, but not all.
A small but not insignificant minority holds otherwise. Whereas these experts agree that dietary saturated fat may raise blood cholesterol, they can provide overwhelming evidence that it is not associated with cardiovascular disease. Almost every large trial has demonstrated this.
Marcia Angell describes the problem in Science on Trial. Angell explains that Frye was not without its critics. Opponents, she wrote, “claimed somewhat improbably, that it would tend to exclude novel, far-sighted testimony by modern-day Galileos. There is no record of this happening once, let alone often.
“Furthermore, even if a modern-day Galileo did not make into court at first, that fact should not stop him from prevailing in the scientific community. Courts do not determine scientific acceptance, as implied by the argument that we need to keep our courts open to the hidden Galileos in our midst.”
But isn’t this exactly what has happened in nutrition and maybe, in general, in the medical community?
The “experts” control editorial boards, granting agencies and academic departments and are as powerful as the Catholic Church in repressing dissent. They have prestige and, in many cases, undisputed accomplishments, but does science run on general acceptance? Does majority (of experts) rule?
One of the problems with Frye that lawyers have addressed is a question of identifying the academic or scientific field in which experts have to consider the general acceptance. Different disciplines hold to different standards.
In the case at hand, biochemists would dismiss out of hand many ideas in nutrition. Physical scientists would consider many methodologies absurd. Intention-to-treat is perhaps the most absurd. The question of who is an expert might have applied to the techniques in the original Frye case, at least as experts might implement it today: “If polygraph examiners are selected as the relevant field, polygraph results would be admissible.”
The epidemic of obesity and diabetes stands as a testament to the failure of the experts.
It’s easy to assemble a small library of books attacking establishment medical nutrition. Uffe Ravnskov has updated his classic Cholesterol Myths in Ignore the Awkward. Gary Taubes’ recent Good Calories, Bad Calories is the most compelling and James Le Fanu’s Rise and Fall of Modern Medicine, the most succinct.
Just sitting at my desk now I can see a dozen others on the bookshelf. Surprisingly, there has been only one rebuttal, Steinberg’s Cholesterol Wars, the subject of the next post .