Healthy Skepticism in Nutrition Science

By this point in the unit, you should understand the types of studies used in nutrition research and the quality of evidence each can provide. As you sift through studies, there are a few limitations of nutrition research that are always worth keeping in mind:

• Challenges in nutrition research: There are some inherent challenges to researching what people eat and how it affects their health. This fact limits the quality of evidence and stirs controversy in nutrition research.
• Influence of food industry funding: Because government funding for nutrition research is limited, much of it is funded by the food industry. Industry’s primary goal is to sell more of their products, so they have a conflict of interest that can result in bias in the way they design studies and interpret the results.

Being alert to these two things means always examining nutrition research with a healthy sense of skepticism. Let’s take a closer look at these two issues in nutrition science.

Challenges in Nutrition Research

How does the food we eat affect our health? This question is exceedingly difficult to answer with certainty. We all need to eat every day, but we can choose from a huge array of possible foods in different combinations. And it’s probably not what we eat on any given day that matters, but what we eat over months and years and decades—our long-term eating patterns—that matter to our long-term health.

Imagine that you’re a nutrition researcher, and you’ve made the observation that over the last 50 years in the U.S., people have been consuming more and more processed foods (foods made with refined ingredients and industrial processes, usually with the addition of sugar, fat, and/or salt). You hypothesize that processed foods are contributing to obesity, which has also increased over the last 50 years. You might first test your hypothesis in animal studies by feeding mice and rats a buffet of potato chips, soda, and Twinkies, and measuring changes in their body weight. You might find that the animals do, in fact, gain weight on this diet. However, you know that what is true in rodents isn’t always true in humans, and you’ll need to study humans in order to understand the role of processed foods in the obesity epidemic.

Your next step might be to conduct an observational study, the most common type of study design in human nutrition research. For example, you might do a cross-sectional study where you compare groups of people who eat a lot of processed foods with those who eat very little. Or you might conduct a prospective cohort study in which you ask people how much processed foods they eat and then follow them over time, looking for correlations between processed food consumption and their body composition.

These types of studies have been conducted, and they’ve found correlations between consumption of processed foods and obesity. For example, a cross-sectional study published in 2018 compared the consumption of processed foods and the prevalence of obesity in 19 European countries and found that countries where people eat more processed foods also have a greater prevalence of obesity.¹ A prospective cohort study published in 2016 followed nearly 8,500 university graduates in Spain and found that those who ate more processed foods were more likely to be overweight or obese 9 years later.²

From these results, can we conclude that eating more processed food causes weight gain? Nope. It’s a tempting conclusion, but this brings us to the first major problem with nutrition research: Observational studies can only show that two variables (eating processed foods and obesity, in this case) are correlated, not that one causes the other. This distinction is especially important in nutrition because diet is intertwined with many other lifestyle and socioeconomic factors that also affect health outcomes. For example, people who eat more processed food might also eat fewer fruits and vegetables, exercise or sleep less, have more stress, or have less access to preventative healthcare. These are just a few of the confounding factors that could explain the observed correlation between processed food consumption and weight gain. Weight gain might have nothing to do with processed food and instead be driven by one or all of these factors, or others that we haven’t considered.

The second major problem with observational nutrition research is that it’s difficult to accurately quantify what and how much people eat, especially over long periods of time. Epidemiological studies usually rely on questionnaires that ask people to remember how much food they ate, but people are notoriously bad at remembering this type of information, and sometimes we fudge the truth. For example, you might remember that you had a cup of coffee but forget that you added cream, completely forget about a mid-morning muffin snack, or guess that you ate 2 cups of veggies when it was closer to 1.5 cups. And many diet questionnaires, called food frequency questionnaires, ask people to recollect and mentally average their food and beverage intake for the last 12 months, not just yesterday. The image below shows part of a page from a 24-page National Health and Nutrition Examination Survey (NHANES) food questionnaire, a national survey often used for research on country-wide dietary patterns. As you can see, these questions are detailed, and there’s plenty of room for small errors to accumulate.

Figure 2.5. A sample page from the NHANES Food Questionnaire.

A third challenge in nutrition research is that diet is just so complex. Stanford physician and researcher John Ioannidis, a frequent critic of observational nutrition research, described the complexity of diet in a 2018 editorial published in JAMA: “Individuals consume thousands of chemicals in millions of possible daily combinations. For instance, there are more than 250,000 different foods and even more potentially edible items, with 300,000 edible plants alone.” He also points out that how an individual responds to a particular dietary pattern can be influenced by genetics, age, and the way they metabolize nutrients. “Disentangling the potential influence on health outcomes of a single dietary component from these other variable is challenging, if not impossible,” Ioannidis wrote.³

Returning to the question of processed foods, all three of these challenges impact how we interpret observational studies that show a correlation between processed food consumption and weight gain. It doesn’t mean that these studies are useless, but we want to be aware of their limitations and consider other ways to test the hypothesis. One way to overcome these challenges is to conduct a randomized controlled trial (RCT), the study design that gives us the highest quality evidence. RCTs are time and funding-intensive experiments, so they’re usually only conducted after consistent evidence from observational and laboratory studies has accumulated.

It turns out that there has been a randomized controlled trial of processed foods and weight gain. It was funded by the National Institutes of Health and published in 2019 in the journal Cell Metabolism.⁴ In this study, 20 participants lived in the NIH’s Clinical Center for one month, where they consumed only processed foods for two weeks and only unprocessed foods for another two weeks, and they could eat as much or as little as they liked during each of these periods. The diets were carefully designed by dietitians so that they were matched in calories, sugar, fat, fiber, and macronutrients, and the exact amounts consumed by the participants were measured every day (solving the problem of measuring diet complexity and accurately describing what and how much people eat every day).

The study found that people ate about 500 kilocalories more and gained about a pound per week when they were eating processed foods. This study design could show causation, not just correlation, because the other nutritional factors like calories, sugar, fat, fiber, and macronutrients were held constant, and the diets were tested in the same people, so other factors such as genetics, sleep, stress, and exercise were constant between the two types of diets. (This was an example of a crossover randomized controlled trial, in which each subject serves as their own control, and they completed the processed and unprocessed phases of the trial in random order.)

Figure 2.6. At left, researchers Kevin Hall and Stephanie Chung talk with one of the processed foods trial participants at the NIH Clinical Center, an inpatient facility where participants lived for the duration of the study. At right, an NIH worker prepares meals for participants in the center’s kitchen. All meals were provided for study participants to carefully control their diet during the trial.

This study suggested something very important—that food processing causes people to eat more food and gain weight. However, even the best study design has limitations. For one thing, this study was small (just 20 participants), and it only lasted for two weeks, so we don’t know if the findings apply to the general population over a lifetime of complex, ever-changing diets. The next steps will be to try to repeat the study in another group of people to see if the finding holds and to design studies to figure out why processed foods cause increased caloric intake.

As you evaluate nutrition research, especially observational studies, keep in mind the inherent challenges of nutrition research and look for randomized controlled trials that can help solve those challenges. Even for randomized controlled trials, consider their limitations, and know that one study is never enough to fully answer a question in the complex field of nutrition.

Industry Influence: Follow the Money

Understanding how diet influences health is a pressing need. By some estimates, a suboptimal diet is the single greatest changeable risk factor contributing to death and disability worldwide, and in the United States, the cost of diet-related chronic diseases are estimated to be as high as $1 trillion each year.⁵ Yet, for all its importance, nutrition science has long suffered from a lack of government investment, with only about 5 percent of the National Institutes of Health (NIH) budget, or $1.8 billion, directed towards research on how the foods we eat affect our health, according to a 2019 investigation by Politico. “In 2018, NIH funding for cancer, which affects just under 9 percent of the population, was $6.3 billion. Funding for obesity, which affects about 30 percent of the country, was about $1 billion,” the article noted.⁶

With so little government funding for nutrition science, who is funding the rest of the studies that feed the constant news cycle? Many are funded by food companies and industry groups, either to conduct studies in their own research divisions or in the form of grants given to university scientists. That’s problematic, because food companies and industry groups have an inherent bias or conflict of interest. Their primary goal is to promote their products and to sell more of them—not to advance knowledge of food and health—and this affects how they frame research questions, design and interpret studies, and report their results.

Marion Nestle, a retired nutrition professor at New York University, has written extensively about this problem. For a year in 2015-2016, she informally tracked industry-funded studies and found that 90 percent of the time, their conclusions benefitted the industry that funded them. In another example, a 2013 meta-analysis found that among studies that looked at whether soda consumption contributed to obesity, those funded by the soda industry were five times more likely to conclude that it doesn’t contribute to obesity compared to those not funded by the industry.⁷ (Consider the processed foods RCT we just discussed. It was funded by the National Institutes of Health, which doesn’t have a stake in the results. Would you trust the results of a study of processed foods if it was funded by Nabisco? Or for that matter, the Broccoli Growers of America? Probably not.)

There’s likely a long history of biased nutrition research influencing dietary advice. For example, in the 1960s, the sugar industry paid well-respected academic scientists to publish research concluding that it was fat—not sugar—that was detrimental to heart health.⁸ (Both too much fat and too much sugar can negatively affect heart health, but it benefited the sugar industry to focus the blame on fat.) As recently as 2015, Coca-Cola was funding research meant to promote lack of physical activity as the main cause of obesity, shifting blame away from dietary factors, such as drinking soda.⁹ When food companies drive the narrative coming out of nutrition research, this can potentially impact public policy.

Media attention has made researchers and policy makers much more aware of the problems with industry funding and conflicts of interest in nutrition research, and they’re working to solve them. But regardless, if you see reports of a study that shows that blueberries can block bladder infections, pistachios can prevent pancreatitis, or cinnamon can cure cancer… well, you should be skeptical, and always check the funding source. Studies on a single type of food are almost always industry-funded.

How to find clarity in a complex field

Let’s review some of the key issues:

• Nutrition research is really difficult to do well. We want to know how nutrition relates to health over the long term, but it’s hard to quantify how people eat over a lifetime and track them for long enough to see an impact.
• We often rely on observational studies, which can only show that two variables are correlated, not that one causes the other.
• Randomized controlled trials are rare, and they’re often small, short-term studies that may or may not tell us what happens in the real world.
• Diet is exceptionally complex, with countless combinations of different nutrients and foods.
• One study is never enough to fully answer a question in the complex field of nutrition.
• Nutrition research is often funded by the food industry, which can be biased towards results that benefit business, not human health.

All of this can be discouraging, but you should also know that researchers are working hard to improve the quality of nutrition research and to interpret it honestly for the rest of us. As a consumer of nutrition information, use a skeptical eye when you read news of the latest nutrition research. Look for areas of consensus, where committees of experts have put their heads together to come up with the best advice they can based on the evidence we have, such as in the Dietary Guidelines for Americans. You’ll find that while experts in this field are often debating the latest controversy, they also agree on a lot. As we continue on in this unit, we’ll talk more about how to find accurate sources of information and who you can trust for evidence-based advice in the field of nutrition.

References

• ¹Monteiro, C. A., Moubarac, J.-C., Levy, R. B., Canella, D. S., Louzada, M. L. da C., & Cannon, G. (2018). Household availability of ultra-processed foods and obesity in nineteen European countries. Public Health Nutrition, 21(1), 18–26. https://doi.org/10.1017/S1368980017001379
• ²Mendonça, R. de D., Pimenta, A. M., Gea, A., de la Fuente-Arrillaga, C., Martinez-Gonzalez, M. A., Lopes, A. C. S., & Bes-Rastrollo, M. (2016). Ultraprocessed food consumption and risk of overweight and obesity: The University of Navarra Follow-Up (SUN) cohort study. The American Journal of Clinical Nutrition, 104(5), 1433–1440. https://doi.org/10.3945/ajcn.116.135004
• ³Ioannidis, J. P. A. (2018). The Challenge of Reforming Nutritional Epidemiologic Research. JAMA, 320(10), 969–970. https://doi.org/10.1001/jama.2018.11025
• ⁴Hall, K. D., Ayuketah, A., Brychta, R., Cai, H., Cassimatis, T., Chen, K. Y., Chung, S. T., Costa, E., Courville, A., Darcey, V., Fletcher, L. A., Forde, C. G., Gharib, A. M., Guo, J., Howard, R., Joseph, P. V., McGehee, S., Ouwerkerk, R., Raisinger, K., … Zhou, M. (2019). Ultra-Processed Diets Cause Excess Calorie Intake and Weight Gain: An Inpatient Randomized Controlled Trial of Ad Libitum Food Intake. Cell Metabolism, 30(1), 67-77.e3. https://doi.org/10.1016/j.cmet.2019.05.008
• ⁵Mozaffarian, D. (2017). Conflict of Interest and the Role of the Food Industry in Nutrition Research. JAMA, 317(17), 1755–1756. https://doi.org/10.1001/jama.2017.3456
• ⁶Boudreau, C., & Evich, H. B. (n.d.). How Washington keeps America sick and fat. POLITICO. Retrieved January 27, 2020, from https://www.politico.com/news/agenda/2019/11/04/why-we-dont-know-what-to-eat-060299
• ⁷Bes-Rastrollo, M., Schulze, M. B., Ruiz-Canela, M., & Martinez-Gonzalez, M. A. (2013). Financial Conflicts of Interest and Reporting Bias Regarding the Association between Sugar-Sweetened Beverages and Weight Gain: A Systematic Review of Systematic Reviews. PLOS Medicine, 10(12), e1001578. https://doi.org/10.1371/journal.pmed.1001578
• ⁸Kearns, C. E., Schmidt, L. A., & Glantz, S. A. (2016). Sugar Industry and Coronary Heart Disease Research: A Historical Analysis of Internal Industry Documents. JAMA Internal Medicine, 176(11), 1680–1685. https://doi.org/10.1001/jamainternmed.2016.5394
• ⁹O’Connor, A. (2015, August 9). Coca-Cola Funds Scientists Who Shift Blame for Obesity Away From Bad Diets. The New York Times. https://well.blogs.nytimes.com/2015/08/09/coca-cola-funds-scientists-who-shift-blame-for-obesity-away-from-bad-diets/

Image Credits

• “Magnifying glass” photo by Emiliano Vittoriosi on Unsplash (license information)
• Figure 2.5.”NHANES food questionnaire” by CDC is in the public domain
• “Table of food photo” by Spencer Davis on Unsplash (license information)
• Figure 2.6. “NIH study center photos” by Jennifer Rymaruk, NIDDK are in the Public Domain