“Inconsistent and contradictory results from nutrition studies conducted by different investigators continue to emerge, in part because of the inherent variability of natural products, as well as the unknown and therefore uncontrolled variables in study populations and experimental designs.” — The Challenge of Reproducibility and Accuracy in Nutrition Research: Resources and Pitfalls
This ad-free article is made possible by the financial support of the
Center for Research on Environmental Chemicals in Humans: a 501(c)(3) non-profit.
Please consider making a tax-deductible donation for continued biomedical research.
Dietary studies are driven by the need and desire for consumers and scientists to determine which foods are the most beneficial for human health. But, unfortunately, diet and health studies are infamous for conflicting results that fall short of credibly causal relationships. Those studies are plagued by equivocal and often contradictory results, lack of specific causality, replication failures, and significant barriers to translating results into valid clinical recommendations. (See: “Further Reading”, below)
As a result, health professionals and their clients are faced with a steady diet of popular articles, such as 5 Foods We Thought Were Bad For Us, Now Turn Out to Be Good.
The same ambivalence spills over into the scientific literature where one study finds evidence for the healthy consumption of nuts, and another is not so sure: Should we go nuts about nuts?
The author comments of one published study exemplifies clinician frustration:
“A significant positive effect of the interventions on weight was reported by all study types. The meta-analysis showed that lifestyle interventions achieved weight and waist circumference reductions after one year. However, no clear effects on biochemical or clinical parameters were observed,… Lifestyle interventions for patients at high risk of diabetes, delivered by a variety of healthcare providers in routine clinical settings, are feasible but appear to be of limited clinical benefit one year after intervention. Despite convincing evidence from structured intensive trials, this systematic review showed that translation into routine practice has less effect on diabetes risk reduction.”
Designing for Causality
Well-designed scientific studies involve a “before and after” protocol. This means that when an experiment is first run, specific outcomes are measured in the “before” state.
Following that only one single variable is changed (such as adding or deleting a single chemical), the experiment is run again. If there is a change in any of the measured outcomes, then a valid conclusion can be drawn that the change in that single chemical is the cause.
However, humans exist in a vast, uncontrollable, constantly changeable, and mostly unknowable matrix of different plastic-based and other chemicals.
In general, most are prospective, retrospective, or epidemiological studies that can offer broad, but non-specific associations of a causal factor which may merit more disciplined examinations. While some may land in a very large ballpark, none succeeds in producing definitively causal relationships.
To approach a level of credible causality requires a clinical trial. These are tightly controlled and intensely monitored experiments with human subjects designed to investigate the health effects or other measured outcomes of specific substances or environmental changes on a target population.
Clinical trial subjects are screened to make sure that they are representative of the appropriately relevant population. Screening also seeks to avoid known confounding factors. Test subjects must then be maintained in environmental conditions that further minimize complications that may bias a causal conclusion.
When properly designed and conducted, clinical trials measure the effects on test subjects of a single independent variable and can produce results that merit a valid decision on whether that independent variable caused a specific measured outcome.
Causality Tests on the Measurement of One Non-Confounded Factor
To accurately claim causality in a dietary intervention study depends on providing the exact same foods, prepared the exact same way, served in exactly the same environmental conditions.
But, because there are many unknowable and co-confounding factors and compounds, exactitude requires maintaining all factors (every food, beverage, and non-food exposure) the same in the before and after legs of a trial with the exception of a single compound that is dosed in the first “contamination” leg.
To assure the most accurate results, meals and beverages for both legs of the trial will be prepared as a single batch before the start of the trial and will be divided in half for each leg.
After establishing an existing state (baseline reading), the first — contamination — leg consists of half of the pre-prepared food and beverages that have been dosed with the substance being evaluated. Following that, the intervention leg consists of the second half of the same food and beverages.
That protocol changes only a single independent variable. As a result, the measured outcomes (hsCRP, Ghrelin, etc.) should therefore accurately warrant a valid conclusion that the change in the independent variable is causal.
Alternatives to Dosing Are Impractical
The application of this classic experimental design to a dietary intervention study is necessary because isolating a relevant independent variable with current dietary intervention protocols is impractical and impossible because it would require mass spectrometry testing of every possible food ingredient and compound in both legs of a study.
This is due to:
- Extreme, and unknowable, variations of food contaminant concentrations from both Plastic-Derived Chemicals and Ultra-Processed Foods additives in most common food items.[i],[ii],[iii],[iv],[v],[vi],[vii]
- Evidence of inherent contamination in production and processing and not solely from food contact materials.[viii],[ix],[x],[xi],[xii],[xiii],[xiv]
- Ubiquitous contamination of all commonly available foods and the impractical need to use extreme procedures for sourcing food. See Appendix 2, and Appendix 3 in the IRB-approved study revision.
- Unknown co-founding interactions of micro-nutrients
Further Reading:
[i] Fasano, E., Bono-Blay, F., Cirillo, T., Montuori, P., and Lacorte, S. 2012. Migration of phthalates, alkylphenols, bisphenol A and di(2-ethylhexyl)adipate from food packaging. Food Control 27(1): 132-138.
[ii] Serrano, S.E., Braun, J., Trasande, L., Dills, R., and Sathyanarayana, S. 2014. Phthalates and diet: a review of the food monitoring and epidemiology data. Environmental Health 13: 43.
[iii] Guart, A., Bono-Blay, F., Borrell, A., and Lacorte, S. 2011. Migration of plasticizers phthalates, bisphenol A and alkylphenols from plastic containers and evaluation of risk. Food Additives & Contaminants Part A: Chemistry, Analysis, Control, Exposure & Risk Assessment 25(5): 676-685.
[iv] Bhunia, K., Sablani, S.S., Tang, J., and Rasco, B. 2013. Migration of chemical compounds from packaging polymers during microwave, conventional heat treatment, and storage. Comprehensive Reviews in Food Science and Food Safety 12(5): 523-545.
[v] Bang, D.Y., Kyung, M., Kim, M.J., Jung, B.Y., Cho, M.C., Choi, S.M., Kim, Y.W., Lim, S.K., Lim, D.S., Won, A.J., Kwack, S.J., Lee, Y., Kim, H.S., and Lee, B.M. 2012. Human risk assessment of endocrine-disrupting chemicals derived from plastic food containers. Comprehensive Reviews in Food Science and Food Safety 11(5): 453-470.
[vi] Groh, K.J., Geuke, B., and Muncke, J. 2017. Food contact materials and gut health: Implications for toxicity assessment and relevance of high molecular weight migrants. Food and Chemical Toxicology 109(1): 1-18.
[vii] Bittner, G.D., Denison, M.S., Yang, C.Z., Stoner, M.A., and He, G. 2014. Chemicals having estrogenic activity can be released from some bisphenol a-free hard and clear, thermoplastic resins. Environmental Health 13: 103.
[viii] Schecter, A., Lorber, M., Guo, Y., Wu, Q., Yun, S.H., Kannan, K., Hommel, M., Imran, N., Hynan, L.S., Cheng, D., Colacino, J.A., and Birnbaum, L.S. 2013. Phthalate concentrations and dietary exposure from food purchased in New York State. Environmental Health Perspectives 121(4): 473-479.
[ix] Cariou, R., Larvor, F., Monteau, F., Marchand, P., Bichon, E., Dervilly-Pinel, G., Antignac, J-P., and Le Bizec, B. 2016. Measurement of phthalates diesters in food using gas chromatography-tandem mass spectrometry. Food Chemistry 196: 211-219.
[x] Van Holderbeke, M., Geerts, L., Vanermen, G., Servaes, K., Sioen, I., De Henauw, S., and Fierens, T. 2014. Determination of contamination pathways of phthalates in food products sold on the Belgian market. Environmental Research 134: 345-352.
[xi] Fasano, E., Bono-Blay, F., Cirillo, T., Montuori, P., and Lacorte, S. 2012. Migration of phthalates, alkylphenols, bisphenol A and di(2-ethylhexyl)adipate from food packaging. Food Control 27(1): 132-138.
[xii] Cirillo, T., Latini, G., Castaldi, M.A., Dipaola, L., Fasano, E., Esposito, F., Scognamiglio, G., Di Francesco, F., and Cobellis, L. 2015. Exposure to di-2-ethyhexyl phthalate, di-N-butyl phthalate and bisphenol A through infant formulas. Journal of Agricultural and Food Chemistry 63(12): 3303-3310.
[xiii] Fierens, T., Vanermen, G., Van Holderbeke, M., De Henauw, S., and Sioen, I. 2012. Effect of cooking at home on the levels of eight phthalates in foods. Food and Chemical Toxicology 50(12): 4428-4435.
[xiv] Ionas, A.C., Dirtu, A.C., Anthonissen, T., Neels, H., and Covaci, A. 2014. Downsides of the recycling process: Harmful organic chemicals in children’s toys. Environment International 65: 54-62.
Further reading
Goals in Nutrition Science 2015–2020
Guasch-Ferré M, Bulló M, Martínez-González MÁ, et al. Frequency of nut consumption and mortality risk in the PREDIMED nutrition intervention trial. BMC Med. 2013;11:164. Published 2013 Jul 16. doi:10.1186/1741-7015-11-164
Rohrmann, S., Faeh, D. Should we go nuts about nuts?. BMC Med 11, 165 (2013). https://doi.org/10.1186/1741-7015-11-165 —
Harnly J. Importance of Accurate Measurements in Nutrition Research: Dietary Flavonoids as a Case Study. Adv Nutr. 2016;7(2):375‐382. Published 2016 Mar 15. doi:10.3945/an.115.010470
Johnston BC, Alonso-Coello P, Bala MM, et al. Methods for trustworthy nutritional recommendations NutriRECS (Nutritional Recommendations and accessible Evidence summaries Composed of Systematic reviews): a protocol [published correction appears in BMC Med Res Methodol. 2019 Dec 17;19(1):240]. BMC Med Res Methodol. 2018;18(1):162. Published 2018 Dec 5. doi:10.1186/s12874-018-0621-8
Garza C, Stover PJ, Ohlhorst SD, et al. Best practices in nutrition science to earn and keep the public’s trust. Am J Clin Nutr. 2019;109(1):225‐243. doi:10.1093/ajcn/nqy337
Ioannidis JPA. Unreformed nutritional epidemiology: a lamp post in the dark forest. Eur J Epidemiol. 2019;34(4):327‐331. doi:10.1007/s10654-019-00487-5
Giovannucci E. Nutritional epidemiology: forest, trees and leaves. Eur J Epidemiol. 2019;34(4):319‐325. doi:10.1007/s10654-019-00488-4
Lappe, Joan M, and Robert P Heaney. “Why randomized controlled trials of calcium and vitamin D sometimes fail.” Dermato-endocrinology vol. 4,2 (2012): 95-100. doi:10.4161/derm.19833
Ioannidis JP. Implausible results in human nutrition research. BMJ. 2013;347:f6698. Published 2013 Nov 14. doi:10.1136/bmj.f6698