Diet myths: The science of what we eat is failing us

World Health Organization recently advised people to avoid using artificial sweeteners for weight loss or to reduce their risk of health issues like heart disease and diabetes. This was based on the agency’s review of available research on artificial sweeteners to date. Unfortunately, people cannot be confident in those findings. That’s because existing studies on artificial sweeteners are plagued by methodological problems. Even the W.H.O. knows this, given that it ultimately described its certainty in the existing evidence as “low.” Maybe it’s true that artificial sweeteners don’t help with weight loss — but we really do not know for sure. This is not a problem reserved for artificial sweeteners alone. The state of nutrition research is poor, and the problems afflict much of the research into dietary and lifestyle claims around things like coffee, wine, dark chocolate, fad diets, the amount you exercise — you name it. This in part explains other recent flip-flopping around whether moderate drinking is “good” for you: A recent review found the research methods used in many past studies on the benefits of drinking alcohol to be flawed.

Diet and exercise are clearly important parts of a healthy lifestyle, but it’s challenging to accurately estimate the specific effect of making any change based on how most nutrition and lifestyle research is currently conducted. Take the case of artificial sweeteners. Randomized studies — where people are randomly assigned to one treatment or another to ensure that no other factors interfere — are considered the gold standard. But randomized trials of sweeteners are often small and brief, which makes it hard to reach reliable conclusions about their long-term effects. The way sweeteners are studied in trials is also often very different from the way people use them in the real world. For example, some trials had participants consume artificial sweeteners in addition to their typical diets, rather than replacing some real sugars in their diets with artificial sweeteners — the intervention researchers are most interested in — often over a period of just a few months.

Many studies, of both sweeteners and other diet and lifestyle behaviors, are not randomized. For example, several studies of sweeteners simply observe people over time, following both their sweetener use and their health outcomes like rates of diabetes or heart attacks. These observational studies, as they’re called, have their own problems, many of which are so serious that it is difficult to take such studies, well, seriously.

The most significant of these problems is well known: Correlation does not imply causation. If people who consume more sweeteners are more likely to have Type 2 diabetes, did the sweeteners cause the diabetes? Or are the people who use more sweeteners also more likely to have diabetes because of other aspects of their diet or health? Researchers can try to account for obvious differences between groups, but it’s impossible to account for everything.

If the typical randomized trials and observational studies of dietary and lifestyle research present so many challenges, how can we get reliable answers?

Reliability still starts with randomization. Randomization is key to establishing cause and effect; it helps make sure two groups are otherwise similar before we consider what happens to those people who consume different amounts of artificial sweeteners, red wine or dark chocolate. In randomized trials, researchers intentionally randomize people to one group or another, but it’s difficult to conduct trials like this that are large enough and long enough to be useful. (Would you let a scientist tell you what to eat every day for the next decade?)

But there are other ways to credibly study the cause-and-effect relationships of dietary and lifestyle behaviors: by identifying situations in which people are exposed to those behaviors, not by the randomizing hands of researchers but by accident. So-called natural experiments, commonly used in economics, are extraordinarily powerful but sorely underused in medical research.

Consider, for example, that in 1953 Britain ended the rationing of sugar and sweets that had been in place since World War II. Interested in studying the effect of sugar intake in early childhood, the economists Paul Gertler and Tadeja Gracner noticed that children born in the years just before the rationing ended spent their infancy and toddler years with limited sugar in their diets because of said rationing. Children born a few years later had early childhood diets heavier in sugar. When those children became adults, their intake of sugar continued to be higher than that of otherwise similar children who were born during sugar rationing.

By measuring the health of these two groups more than 50 years later — far longer than any clinical trial could reasonably follow people — the economists found that the additional sugar intake led to higher rates of diabetes, elevated cholesterol, arthritis and measures of chronic inflammation. Another way people can be accidentally randomized to health behaviors is through their genes. Consider the heavily studied question of whether alcohol, in moderation, is “good” or “bad” for your health. In a study of over 500,000 Chinese adults, researchers took advantage of the genetic variations that cause some adults, randomly, to enzymatically process alcohol differently, leading to unpleasant symptoms such as flushing. Because those individuals tend to drink less alcohol, researchers can study the causal relationship between alcohol use and health outcomes by examining otherwise similar people with and without specific gene variants, an approach called Mendelian randomization.

Medical researchers pressured professionally to “publish or perish” are often incentivised to publish simple observational studies that lack empirical rigour. Medical journals, responding to public interest in information about diet, in turn encourage this research despite knowing its significant limitations. Media coverage may simply add to the confusion.

The now decades-old “credibility revolution” in economics advanced the use of high-quality, often creative research designs in empirical economic work — so much so that in 2021, a group of economists were awarded the Nobel Prize for their work with natural experiments. Although medical researchers are increasingly taking advantage of natural experiments — thanks in part to large increases in digital data in recent years — these methods remain under-taught and underused, particularly when it comes to diet. This important research needs a credibility revolution of its own.

Dr. Jena and Dr. Worsham are researchers at Harvard Medical School and the authors of “Random Acts of Medicine...”