The Perils of Nutritional Research
Americans are eating more red meat than ever. Maybe that is okay, because just a few months ago we were told that a new and very impressive series of reviews of nutritional studies concluded there is very little evidence that red meat consumption is harmful to human health.
Almost immediately after that study was published, however, waves of criticism appeared from equally impressive sources, insisting the methods used to analyze the data were inappropriate and the conclusion that red meat is safe to eat unjustified. Red meat, like eggs, butter, diet soft drinks, and so many other things we eat seem to be safe one day and dangerous the next, as each new study contradicts the findings of the last. Is it any wonder that many people just throw up their hands and decide that the vagaries of nutritional science make it too fickle to accept? Why not just eat whatever tastes good and hope that someday the scientists work out their differences and give us some definitive diet advice?
A series of reviews and recommendations published last October indicates that the evidence that red meat is a health harm is very weak, but other organizations and experts immediately demurred (source: Shutterstock).
The “controversy” about whether red meat consumption began last October with the publication of a paper in the Clinical Guidelines section of the highly respected and peer reviewed journal Annals of Internal Medicine. The paper details the rigorous steps an international panel of experts took in conducting five separate reviews of studies on the health effects of both unprocessed and processed red meat. The paper includes steps for dealing with potential conflicts of interest among panel members and a table in an appendix listing the members’ individual declarations of conflicts of interest.
The review concludes there is only “low- to very low-certainty evidence” that diets high in red meat increase the “risk for major cardiometabolic outcomes and cancer mortality and incidence.” The same conclusion was reached for processed meats, leading to the final recommendation that “people continue their current meat consumption …”
Red Meat Pushback Is Fierce
This recommendation contradicts those of the World Health Organization, the American Heart Association, and the U.S Department of Health and Human Services. The pushback against these findings and recommendations was fierce. As Julia Belluz wrote in Vox:
“…it comes as no surprise that already, the Annals series has prompted a fierce blowback from various groups who’ve long argued that red and processed meat consumption should be curbed. The American Cancer Society, American Heart Association, Harvard T.H. Chan School of Public Health, and a slew of other researchers objected to the series. The Physicians Committee for Responsible Medicine — a group that’s long endorsed a plant-based diet — filed a petition with the Federal Trade Commission in response to the studies, asking the agency to ‘correct false statements’ contained in the report, which they deemed a ‘major disservice to public health’.
The New York Times even attempted to impugn the panel’s findings and recommendations by asserting that “its lead author has past research ties to the meat and food industry”. Apparently, Bradley C. Johnston, an epidemiologist at Dalhousie University in Canada, failed to disclose in the article that he had received funding for a study on sugar consumption in 2016 that was funded by, according to the New York Times, the:
“International Life Sciences Institute, or ILSI, an industry trade group largely supported by agribusiness, food and pharmaceutical companies and whose members have included McDonald’s, Coca-Cola, PepsiCo and Cargill, one of the largest beef processors in North America. The industry group, founded by a top Coca-Cola executive four decades ago, has long been accused by the World Health Organization and others of trying to undermine public health recommendations to advance the interests of its corporate members.”
The article in the New York Times tries very hard to discredit the findings of the international panel but misstates the panel’s recommendations in the very first sentence when it says it “gave consumers the green light to eat more red and processed meat.” In fact, the panel’s recommendations clearly state that people should “continue their current meat consumption” and nowhere give any “green light” to eat more red or processed meat. Furthermore, the Times article does not present any methodological or data-based reasons upon which to actually doubt the results of the reviews and recommendations other than the fact that one investigator admitted to making a mistake in neglecting to disclose a prior relationship. Everyone agrees that it is critical to disclose all potential and real conflicts of interest in research papers but failing to do so does not automatically make the findings incorrect.
The Complexities of Nutritional Research
What the red meat controversy highlights is the constant shifting in the guidance we receive from experts about what constitutes a healthy diet. We think it is very important, therefore, that everyone understand why nutritional research is especially prone to uncertainty.
There are three basic ways that we can test the effects of specific types of food on health. The first are animal studies in which any of a variety of species of animal other than humans is fed a rigorously controlled diet and the incidence of things like heart disease and tumors is carefully recorded. Such studies are very important, but of course hard to translate to humans, first because people don’t eat carefully controlled diets for long periods of time and second because physiology differs profoundly between rats and humans.
The second type of nutritional studies are randomized controlled studies in humans. In such studies, two groups of people receive exactly the same food to eat except for one difference, like high versus low portions of red meat. Because the experimenters give all the meals to the people in the study, there is some certainty as to what they are actually eating. Once again, health outcomes are measured. But despite the controlled conditions of such a study, there are still many problems. Study subjects are asked to record how much of the provided meals they actually eat and whether they eat anything not provided for by the researchers. People are notoriously unreliable, however, in filling out such food surveys, including failing to admit when they “cheat.”
Another problem is that studies like this cannot be conducted for long periods of time: the practicalities of providing regular, daily meals to a large group of study subjects make such studies expensive and unwieldy, so they cannot go much beyond weeks to a few months. But the health effects of one’s diet may take many years to become apparent. Few heart attacks or new cancer diagnoses will show up in a group of people eating a lot of red meat for just a month. In an op-ed piece in the November 13, 2019 New York Times, nutrition scientists David S. Ludwig and Steven B. Heymsfield noted “For these reasons, short term trials may have little relevance to understanding how diet affects health over the long term”.
Nutritional studies attempt to draw causal relationships between things we eat and health outcomes like heart disease and cancer, but very often fall short of that call and create confusion about what exactly is a healthy diet (source: Shutterstock).
In order to study the long-term effects of different diets, researchers must therefore turn to a third form of study, the observational study. Here, people are asked to fill out daily food diaries for long periods of time and after some period of years these are analyzed to see if, statistically, there is an association between a specific nutrient or type of food and an adverse health outcome. As mentioned above, asking people to remember what they ate and record it accurately is a notoriously unreliable research strategy. Perhaps even more worrisome is that such uncontrolled studies, although having the virtue of being conducted for longer periods of time than randomized controlled studies, cannot account fully for a myriad of other lifestyle and medical variables, such as how much people in the study exercise, smoke, drink alcohol, or take medications. These are likely to vary within individuals of the many years of study, making them extremely difficult to take into account when analyzing the data at the end of the study. Perhaps, for example, people who eat a lot of red meat also exercise less and smoke cigarettes more. While such things are also recorded on surveys, it is nearly impossible in long-term studies to be absolutely certain exactly which element is responsible for any health outcomes detected.
Finally, when a statistical association from an observational study is found between a nutrient—like red meat—and a bad outcome—like heart disease, it is often a very small association, or as statisticians say, a very small effect size. And that is exactly what the panel who conducted the reviews on red meat consumption published in the Annals of Internal Medicine are saying: while there may be a statistical association between eating red meat and developing heart disease or cancer, it is such a small effect as to make it unimportant.
In discussing the red meat controversy, cardiologist Anthony Pearson writes “…the science behind most nutritional recommendations is weak, and public health authorities often make sweeping dietary recommendations that aren’t justified”. Research studies that are funded by the National Institutes of Health (NIH) or are conducted by companies seeking drug approvals from the Food and Drug Administration (FDA) must be publicly registered at ClinicalTrials.gov. Even before entering subjects into a study, investigators must state what their primary and secondary outcomes are, that is, exactly what outcomes they intend to measure once the data are all collected. But in a study published last month, it was revealed that diet studies are four times more likely to have discrepancies between the main outcomes the investigators said they were going to address before starting the study and what they actually analyze and report when the study results are finally published in a scientific journal. This highlights the instability and potential unreliability of nutrition studies.
John P. A. Ioannidis of the Meta-Research Innovation Center at Stanford University, a frequent critic of nutritional research methods and conclusions, recently wrote: “Observational studies touting small effects despite high risk of confounding and selection biases should rarely be published by general medical journals. These reports should be placed in specialist journals with proper acknowledgement of their limitations in the Abstract…Regardless of where they get published, these articles should not be accompanied by press releases. News media should substantially reduce coverage of such studies; doing this will help strengthen their reputation for seriousness”.
Not all nutritional and lifestyle research is unimportant. There are several things these studies have shown us that now meet high standards of reliability and scientific consensus. These include recommendations to eat more fruits, vegetables, and grains; to not become overweight or obese; to not smoke combustible tobacco products or drink excessive amounts of alcohol; and to get regular exercise. There are also other reasons to shun red meat besides health. Some people have concerns about animal welfare and it is clear that meat production is bad for the environment. The latter fact is perhaps the most compelling reason why we should all curtail our red meat consumption and eat a more plant-based diet.
If we really understand the pitfalls inherent in nutritional research, perhaps we will be able to accept the several recommendations that are solid and be less confused by the ones that are in question. Nutritional science is not problematic because scientists are sloppy or can’t agree on anything. Rather, and for the reasons explained above, the effect on health of what we eat is fundamentally a difficult thing to study. Some data suggest that red meat is associated with an increased risk for heart disease and cancer and others do not. When you put them all together, the signal becomes very small and sometimes even vanishes altogether. More important than dwelling on the health risks of red meat, we should be emphasizing its environmental risks. What we also must do is continuously dwell on the things we know for sure are big health risks. It may be boring to remind people over and over again that obesity, cigarette smoking, lack of regular exercise, binge alcohol consumption, and unprotected sun exposure are all clearly dangerous. That is why we don’t read newspaper articles or see social media posts about these things very often, but rather are treated to endless alerts about every new and dramatic piece of diet research that seems to contradict what we’ve been told before.
If you smoke cigarettes, are sedentary, or eat so much sugar-laden food that you become overweight it really makes little difference how much steak one consumes: poor health is almost guaranteed from such choices. So let’s concentrate on what really puts our health and the environment at risk and give less attention to the endless controversies about everything else.