The Study

Navigation

Significant Changes Necessary in Order for Dietary Intervention Studies to be Causal and Replicable

By on May 22, 2020 in Featured, The Science

Dietary intervention studies as a whole are inherently flawed and do not produce causal or clinically relevant health recommendations or decisions.

This ad-free article is made possible by the financial support of the
Center for Research on Environmental Chemicals in Humans: a 501(c)(3) non-profit.
Please consider making a tax-deductible donation for continued biomedical research.

Accuracy and replicability of any study depend upon using precise methods under identical conditions using identical materials, reagents, apparatus, and test subjects. Unfortunately, human dietary intervention studies in the scientific literature are fatally confounded by the failure to employ basic scientific methods that are standard in bench experiments.

Further inaccuracies are introduced because dietary intervention studies relegate food preparation to caterers, institutional kitchens, or even test subject memories. The kitchen is treated as a black box from which meals emerge without the level of documentation required in standard bench lab investigations. (See: Black Box meals: A look at five dietary intervention studies. )

Most studies involving BPA and phthalates have given general instructions to food preparers to use only “fresh” ingredients and to avoid any plastic contact. However, specific food sources and the details of their production are not available. Neither are specific cooking protocols nor the identities of the utensils, pots, pans, cleaning regimen (surfactants an issue), recipes, and ingredients (processing & methyl contributors an issue).

Data regarding those conditions cannot account for the wide variability of contamination in the growing, processing, and sourcing stages. As a result, instructions in previous dietary studies account only for contamination transfer from food packaging materials and preparation.

No studies have been located in the published literature that succeed in quantitatively parsing the growing, production, and processing contamination from that of food contact materials and preparation.

Unlike clinical blood panels and mass spectrometer tests which have accepted standards, meals — even those using the same recipes — are rife with confounding factors to which no laboratory-quality standards have ever been addressed.

Few chefs, other than bakers, adhere to the precise requirements of recipes. And even bakers can be confounded by variations in measurements, sourced materials, timing, and baking temperatures.

In addition, non-food exposures, must be strictly controlled because PM2.5 particles and other pollutants will influence outcomes. This requires the use of a dormitory environment.

Replication is not causation

But while replicability and accuracy are increased by the use of scientific standards and food-related laboratory best practices, controlling non-food exposures and eliminating the black box only gets a study into the realm of replicability.

Causality, on the other hand requires:

  1. Accounting for unknown interactions of co-confounding factors
  2. Dosing of a single compound
  3. Direct human experimentation not extrapolation from animal models

Accounting for Unknown interactions of co-confounding factors prohibit valid causality conclusions

Dietary interventions as now conducted are incapable of rendering valid causal conclusions about specific substances, compounds, or chemicals. This is because the near-infinite variations in foods produce unknown and unknowable co-contamination errors that are further confounded by preparation inconsistencies.

This can be illustrated by comparing two recent dietary intervention studies — one studying Plastic-Derived Chemicals (Reus, Perdue) and the second of Ultra-Processed Foods (NIH/Hall). While the two studies were quite different in scope, size, and duration, they arrived at very similar results in the behavior of the high-sensitivity C-Reactive Protein (hsCRP) test as a clinical measure of intervention inflammation changes.

The NIH/Hall study, “Ultra-Processed Diets Cause Excess Calorie Intake and Weight Gain,” was a 28-day trial published in the May 16, 2019 issue of Cell Metabolism and represented the most thoroughly documented dietary intervention published so far. In addition to documenting the association between Ultra-Processed Foods (UPFs) and weight gain from over-eating, the study was conducted in a dormitory setting to compensate for non-food environmental exposures, test subject compliance and differences in exercise, and metabolism. In addition, the study conducted an expensive battery of clinical tests (op. cit. Table 3) including hsCRP, insulin, and the appetite peptides leptin, ghrelin, and adiponectin.

The Reus, Perdue six-day trial was far more limited in scope and sought primarily to determine whether hsCRP levels could be affected by Bisphenol A and other commonly associated Plastic-Derived Chemicals (PDCs). Extensive efforts conducted by investigators over a period of more than three years  to determine which foods, sources, packaging, and preparation methods were least — and most — contaminated by PDCs. That work included efforts to:

Among the data and methods developed by the Reus, Perdue investigators was the determination that fast, convenient, ready-to eat, and pre-prepared — Ultra-Processed — foods were the most contaminated by PDCs. For that reason UPFs were often chosen for the “contamination” leg of the study. Intervention leg meals were “reverse engineered” to match the composition of the UPFs in order to make the “before and after” legs as identical as possible save for PDC contamination.

Within days of the publication of the NIH/Hall study, Reus, Perdue investigators initially considered replicating the  “before and after” selections of that extensive study and requested recipes from the investigators. No response was ever made. A thorough examination of supplemental material from NIH/Hall failed to find recipes or any food sourcing and preparation details.

In the interim, a closer examination by the Reus, Perdue investigators revealed that the NIH/Hall intervention meals included foods that they had previously determined had been processed more than minimally and likely contained higher PDC levels than desired.

Similar results and an anomaly indicate unavoidable mutual co-contaminants that confounded both dietary interventions and negated causal conclusions

Levels of hsCRP decreased 21.4% from baseline to end of the minimally contaminated leg: 1.1 mg/L from 1.4 mg/L. In addition, our study results demonstrated a final reduction in hsCRP that was approximately half of that reposted by a major NIH-funded dietary intervention by Hall, et. al. published in 2019 approximately four months prior to our trial.

NIH/Hall data from: “Ultra-Processed Diets Cause Excess Calorie Intake and Weight Gain: An Inpatient Randomized Controlled Trial of Ad Libitum Food Intake”

The lower overall decrease in hsCRP in the Reus, Perdue study was reasonably attributable to:

  • The shorter duration of our trial (six days rather than 28), and
  • The Northern California wildfires of November/December 2019, the smoke pollution from which dramatically increased PM2.5 particulate which is a notorious promoter of inflammation.

Further discussion is available in the Reus, Perdue study.

Investigating an Unexpected Anomaly of the Intervention Legs

Despite expectations that inflammation would increase with foods projected to contain higher levels of BPA, hsCRP actually declined at the end of the “typical/contaminated” diet leg. Significantly,  NIH/Hall independently exhibited this same anomaly.

Efforts to explain this anomaly led investigators to consider possible causes.

NIH/Hall’s primary outcome focused on obesity and investigated weight gain from the ad libitum consumption of UPFs versus less processed foods. The results confirmed weight gain from the contaminated UPF leg of the study.

Since Bisphenol A has been widely accepted as an “obesegen<Google Scholar cite>“, and is associated with obesity, diabetes, insulin dependence as well as cancer, and cardiovascular disease, it may possibly explain – at least in part – the anomaly of how BPA and hsCRP counterintuitively declined at the end of the “typical/contaminated” diet.

Further, chronic inflammation is associated with all of those syndromes and non-communicable diseases. That was a primary reason Reus, Perdue study investigators had chosen hsCRP as a marker for PDC concentrations.

The NIH/Hall study included the “appetite peptides” ghrelin, leptin, and adiponectin are known to be associated with eating behavior. Because of that, the Reus, Perdue investigators searched the literature for relevant BPA clinical markers of health outcome associations.

Data revealed that Bisphenol A affected ghrelin, leptin, and adiponectin in the ways the NIH/Hall study recorded for Ultra-Processed Foods.

In addition, the metabolic peptide behavior in the NIH/Hall study is the same as those caused by BPA exposure:

A further search for disease/syndromes and clinical indicator similarities between Plastic-Derived Chemicals and Ultra-Processed Foods (table below) further confirmed that both show nearly identical activity. From this, the only conclusion that can be reached is that neither trial can delineate a specific causal action of PDC’s or UPFs because both are present in study foods, exert almost identical effects and are, therefore, mutually confounding factors.

The path to a revised protocol: Confounding factors lead to greater complexity and expense

From the beginning, this study’s investigators recognized that basic nutritional levels — protein, fat, and sugar/carbohydrates would need to be the same at each level of the study. While unnecessary in previous dietary intervention studies that measured only BPA levels, the present study also measures health effects which can be affected by nutritional composition.

The widespread variance in BPA concentrations in source foods led to the realization that the reproducibility of this study would require testing of all source food for BPA concentrations. Otherwise, any record of increase or decrease in subject BPA concentrations would be invalid.

That recognition led to the conclusion that a reproducible investigation required the establishment of specific baseline food concentrations — leading to accurate dose levels of BPA consumed.

Investigators’ further research indicated that the American food chain is contaminated at every level by BPA and phthalates. The resulting food sourcing and testing demands increased the costs of the study far beyond the previous budget and would require extraordinary — and expensive — food production and processing methods to be implemented at the farm level and continuing in a highly controlled regime through processing, and preparation for table consumption.

This would require massive financial and personnel resources that make any such study impractical.

Dosing of a single compound is necessary to establish causality in dietary interventions & enable clinically valid health decisions.

“Inconsistent and contradictory results from nutrition studies conducted by different investigators continue to emerge, in part because of the inherent variability of natural products, as well as the unknown and therefore uncontrolled variables in study populations and experimental designs.” — The Challenge of Reproducibility and Accuracy in Nutrition Research: Resources and Pitfalls

Dietary studies are driven by the need and desire for consumers and scientists to determine which foods are the most beneficial for human health. But, unfortunately, diet and health studies are infamous for conflicting results that fall short of credibly causal relationships. Those studies are plagued by equivocal and often contradictory results, lack of specific causality, replication failures, and significant barriers to translating results into valid clinical recommendations. (see: “further reading” below)

As a result, health professionals and their clients are faced with a steady diet of popular articles like this one: 5 Foods We Thought Were Bad For Us, Now Turn Out to Be Good

The same ambivalence spills over into the scientific literature where one study finds evidence for the healthy consumption of nuts, and another is not so sure: Should we go nuts about nuts?

The author comments of one published study exemplifies clinician frustration:

“A significant positive effect of the interventions on weight was reported by all study types. The meta-analysis showed that lifestyle interventions achieved weight and waist circumference reductions after one year. However, no clear effects on biochemical or clinical parameters were observed,… Lifestyle interventions for patients at high risk of diabetes, delivered by a variety of healthcare providers in routine clinical settings, are feasible but appear to be of limited clinical benefit one year after intervention. Despite convincing evidence from structured intensive trials, this systematic review showed that translation into routine practice has less effect on diabetes risk reduction.”

Designing for Causality

Well-designed scientific studies involve a “before and after” protocol.

First, an experiment is conducted and specific outcomes are measured in the “before” state.

Following that only one single variable is changed (such as adding or deleting a single chemical), the experiment is run again. If there is a change in any of the measured outcomes, then a valid conclusion can be drawn that the change in that single chemical is the cause.

However, humans exist in a vast, uncontrollable, constantly changeable, and mostly unknowable matrix of different plastic-based and other chemicals.

In general, most are prospective, retrospective, or epidemiological studies that can offer broad, but non-specific associations of a causal factor which may merit more disciplined examinations. While some may land in a very large ballpark, none succeeds in producing definitively causal relationships.

To approach a level of credible causality requires a clinical trial. These are tightly controlled and intensely monitored experiments with human subjects designed to investigate the health effects or other measured outcomes of specific substances or environmental changes on a target population.

Clinical trial subjects are screened to make sure that they are representative of the appropriately relevant population. Screening also seeks to avoid known confounding factors. Test subjects must then be maintained in environmental conditions that further minimize complications that may bias a causal conclusion.

Properly designed and conducted clinical trials measure the effects on test subjects of a single independent variable and can produce results that merit a valid decision on whether that independent variable caused a specific measured outcome.

Causality Tests on the Measurement of One Non-Confounded Factor

To accurately claim causality in a dietary intervention study depends on providing the exact same foods, prepared the exact same way, served in exactly the same environmental conditions.

Because there are many unknowable and co-confounding factors and compounds, exactitude requires maintaining all factors (every food, beverage, and non-food exposure) the same in the before and after legs of a trial with the exception of a single compound that is dosed in the first “contamination” leg.

To assure the most accurate results, meals and beverages for both legs of the trial should be prepared as a single batch before the start of the trial and then be divided in half for each leg.

After establishing an existing state (baseline reading), the first — contamination — leg consists of half of the pre-prepared food and beverages that have been dosed with the substance being evaluated. Following that, the intervention leg consists of the second half of the same food and beverages.

That protocol changes only a single independent variable. As a result, the measured outcomes (hsCRP, Ghrelin, etc.) should therefore accurately warrant a valid conclusion that the change in the independent variable is causal.

Alternatives to Dosing Are Impractical

The application of this classic experimental design to a dietary intervention study is necessary because isolating a relevant independent variable with current dietary intervention protocols is impractical and impossible because it would require mass spectrometry testing of every possible food ingredient and compound in both legs of a study.

This is due to:

  1. Extreme, and unknowable, variations of food contaminant concentrations from both Plastic-Derived Chemicals and Ultra-Processed Foods additives in most common food items.[i],[ii],[iii],[iv],[v],[vi],[vii]
  2. Evidence of inherent contamination in production and processing and not solely from food contact materials.[viii],[ix],[x],[xi],[xii],[xiii],[xiv]
  3. Ubiquitous contamination of all commonly available foods and the impractical need to use extreme procedures for sourcing food. See Appendix 2, and  Appendix 3 in the IRB-approved study revision.
  4. Unknown co-founding interactions of micro-nutrients.

 

[i] Fasano, E., Bono-Blay, F., Cirillo, T., Montuori, P., and Lacorte, S. 2012. Migration of phthalates, alkylphenols, bisphenol A and di(2-ethylhexyl)adipate from food packaging. Food Control 27(1): 132-138.

[ii] Serrano, S.E., Braun, J., Trasande, L., Dills, R., and Sathyanarayana, S. 2014. Phthalates and diet: a review of the food monitoring and epidemiology data. Environmental Health 13: 43.

[iii] Guart, A., Bono-Blay, F., Borrell, A., and Lacorte, S. 2011. Migration of plasticizers phthalates, bisphenol A and alkylphenols from plastic containers and evaluation of risk. Food Additives & Contaminants Part A: Chemistry, Analysis, Control, Exposure & Risk Assessment 25(5): 676-685.

[iv] Bhunia, K., Sablani, S.S., Tang, J., and Rasco, B. 2013. Migration of chemical compounds from packaging polymers during microwave, conventional heat treatment, and storage. Comprehensive Reviews in Food Science and Food Safety 12(5): 523-545.

[v] Bang, D.Y., Kyung, M., Kim, M.J., Jung, B.Y., Cho, M.C., Choi, S.M., Kim, Y.W., Lim, S.K., Lim, D.S., Won, A.J., Kwack, S.J., Lee, Y., Kim, H.S., and Lee, B.M. 2012. Human risk assessment of endocrine-disrupting chemicals derived from plastic food containers. Comprehensive Reviews in Food Science and Food Safety 11(5): 453-470.

[vi] Groh, K.J., Geuke, B., and Muncke, J. 2017. Food contact materials and gut health: Implications for toxicity assessment and relevance of high molecular weight migrants. Food and Chemical Toxicology 109(1): 1-18.

[vii] Bittner, G.D., Denison, M.S., Yang, C.Z., Stoner, M.A., and He, G. 2014. Chemicals having estrogenic activity can be released from some bisphenol a-free hard and clear, thermoplastic resins. Environmental Health 13: 103.

[viii] Schecter, A., Lorber, M., Guo, Y., Wu, Q., Yun, S.H., Kannan, K., Hommel, M., Imran, N., Hynan, L.S., Cheng, D., Colacino, J.A., and Birnbaum, L.S. 2013. Phthalate concentrations and dietary exposure from food purchased in New York State. Environmental Health Perspectives 121(4): 473-479.

[ix] Cariou, R., Larvor, F., Monteau, F., Marchand, P., Bichon, E., Dervilly-Pinel, G., Antignac, J-P., and Le Bizec, B. 2016. Measurement of phthalates diesters in food using gas chromatography-tandem mass spectrometry. Food Chemistry 196: 211-219.

[x] Van Holderbeke, M., Geerts, L., Vanermen, G., Servaes, K., Sioen, I., De Henauw, S., and Fierens, T. 2014. Determination of contamination pathways of phthalates in food products sold on the Belgian market. Environmental Research 134: 345-352.

[xi] Fasano, E., Bono-Blay, F., Cirillo, T., Montuori, P., and Lacorte, S. 2012. Migration of phthalates, alkylphenols, bisphenol A and di(2-ethylhexyl)adipate from food packaging. Food Control 27(1): 132-138.

[xii] Cirillo, T., Latini, G., Castaldi, M.A., Dipaola, L., Fasano, E., Esposito, F., Scognamiglio, G., Di Francesco, F., and Cobellis, L. 2015. Exposure to di-2-ethyhexyl phthalate, di-N-butyl phthalate and bisphenol A through infant formulas. Journal of Agricultural and Food Chemistry 63(12): 3303-3310.

[xiii] Fierens, T., Vanermen, G., Van Holderbeke, M., De Henauw, S., and Sioen, I. 2012. Effect of cooking at home on the levels of eight phthalates in foods. Food and Chemical Toxicology 50(12): 4428-4435.

[xiv] Ionas, A.C., Dirtu, A.C., Anthonissen, T., Neels, H., and Covaci, A. 2014. Downsides of the recycling process: Harmful organic chemicals in children’s toys. Environment International 65: 54-62.

Further reading

Goals in Nutrition Science 2015–2020

Reduction of diabetes risk in routine clinical practice: are physical activity and nutrition interventions feasible and are the outcomes from reference trials replicable? A systematic review and meta-analysis

Guasch-Ferré M, Bulló M, Martínez-González MÁ, et al. Frequency of nut consumption and mortality risk in the PREDIMED nutrition intervention trial. BMC Med. 2013;11:164. Published 2013 Jul 16. doi:10.1186/1741-7015-11-164

Rohrmann, S., Faeh, D. Should we go nuts about nuts?. BMC Med 11, 165 (2013). https://doi.org/10.1186/1741-7015-11-165 —

Harnly J. Importance of Accurate Measurements in Nutrition Research: Dietary Flavonoids as a Case Study. Adv Nutr. 2016;7(2):375‐382. Published 2016 Mar 15. doi:10.3945/an.115.010470

Johnston BC, Alonso-Coello P, Bala MM, et al. Methods for trustworthy nutritional recommendations NutriRECS (Nutritional Recommendations and accessible Evidence summaries Composed of Systematic reviews): a protocol [published correction appears in BMC Med Res Methodol. 2019 Dec 17;19(1):240]. BMC Med Res Methodol. 2018;18(1):162. Published 2018 Dec 5. doi:10.1186/s12874-018-0621-8

Garza C, Stover PJ, Ohlhorst SD, et al. Best practices in nutrition science to earn and keep the public’s trust. Am J Clin Nutr. 2019;109(1):225‐243. doi:10.1093/ajcn/nqy337

Johnston BC, Seivenpiper JL, Vernooij RWM, et al. The Philosophy of Evidence-Based Principles and Practice in Nutrition. Mayo Clin Proc Innov Qual Outcomes. 2019;3(2):189‐199. Published 2019 May 27. doi:10.1016/j.mayocpiqo.2019.02.005
Satija A, Stampfer MJ, Rimm EB, Willett W, Hu FB. Perspective: Are Large, Simple Trials the Solution for Nutrition Research?. Adv Nutr. 2018;9(4):378‐387. doi:10.1093/advances/nmy030

Ioannidis JPA. Unreformed nutritional epidemiology: a lamp post in the dark forest. Eur J Epidemiol. 2019;34(4):327‐331. doi:10.1007/s10654-019-00487-5

Giovannucci E. Nutritional epidemiology: forest, trees and leaves. Eur J Epidemiol. 2019;34(4):319‐325. doi:10.1007/s10654-019-00488-4

Lappe, Joan M, and Robert P Heaney. “Why randomized controlled trials of calcium and vitamin D sometimes fail.” Dermato-endocrinology vol. 4,2 (2012): 95-100. doi:10.4161/derm.19833

Ioannidis JP. Implausible results in human nutrition research. BMJ. 2013;347:f6698. Published 2013 Nov 14. doi:10.1136/bmj.f6698

Blumberg J, Heaney RP, Huncharek M, Scholl T, Stampfer M, Vieth R, et al. Evidence-based criteria in the nutritional context. Nutr Rev. 2010;68:478–84. doi: 10.1111/j.1753-4887.2010.00307.x. [PubMed]
Heaney RP. Nutrients, endpoints, and the problem of proof. J Nutr. 2008;138:1591–5. [PubMed]
Heaney RP, Weaver CM, Blumberg J. EBN (Evidence-Based Nutrition) Ver. 2.0. Nutr Today. 2011;46:22–6. doi: 10.1097/NT.0b013e3182076fdf. [CrossRef] 
Biesalski HK, Aggett PJ, Anton R, Bernstein PS, Blumberg J, Heaney RP, et al. 26th Hohenheim Consensus Conference, September 11, 2010 Scientific substantiation of health claims: evidence-based nutrition. Nutrition. 2011;27(Suppl):S1–20. doi: 10.1016/j.nut.2011.04.002. [PubMed]

 

Causality and clinical usefulness of dietary interventions require direct human experimentation not extrapolation from animal models

Human dietary studies and interventions are often clinically frustrating because they frequently fail to provide consistent, human-relevant causal results. In addition, human studies are expensive and require strict ethics standards. Complicating this is the fact that — even with animal studies — there has been no consistent set of protocols and lab standards for conducting those trials.

This chaos and inconsistency leads one substantial study (Carra, p.9) to declare:  “With this type of data, it is difficult to draw a meaningful, intellectually honest conclusion.”

Significantly,a search of the scientific literature failed to locate controlled clinical direct human studies measuring clinically significant health effects of low-level Plastic-Derived Chemical (PDC) contamination. However, a literature search did reveal a handful of investigations in which human test subjects were directly administered Bisphenol A in order to assess BPA pharmacokinetics (but not health effects).

Compounding that lack of health-effects knowledge, human dietary intervention studies published so far on BPA and other plastic chemicals have focused only measuring levels of marker chemicals and have not measured direct, clinically relevant health outcome indicators.

Extrapolation of human health effects from murine model mostly fails and creates controversy

The absence of clinically relevant human studies required that human health outcome conclusions require  extrapolation from murine models.

However, no clinically credible conclusions about causation can be drawn from that extrapolation because murine model results frequently fail to translate accurately to humans. This is especially the case in drug trials where up to 92% have been shown to fail in humans following successful murine results.

That situation has, in turn, contributed to an internationally divisive scientific controversy that has prevented health professionals and consumers from making scientifically valid health decisions.

On one side of the controversy are university and independent scientists who contend that hundreds of published studies indicate that micro- and pico-molar concentrations of BPA are unhealthy. Three many examples include:

On the other side, corporate and some federal regulatory scientists point to the recent CLARITY-BPA study whose results — they contend — demonstrated that low levels are safe.

National Toxicology Program. 2018. NTP Research Report on the CLARITY-BPA Core Study: A Perinatal and Chronic Extended-Dose-Range Study of Bisphenol A in Rats. NTP RR 9. Research Triangle Park, NC. National Toxicology Program (9): 1-221.

CLARITY-BPA and most other published studies have been based upon an in vivo murine model whose accurate extrapolation to humans is not known. As a result, the current debate centers on protocol flaws, confounding factors, sources of contamination and the finer points of how the murine studies were conducted.

CLARITY-BPA Conclusions Challenged

Further, the CLARITY-BPA study and its predecessor are plagued by serious and scientifically fatal protocol errors that invalidate any conclusions regarding human safety or health:

Resistance to human dosing: Nuremberg’s legacy?

The lack of controlled-dose human studies has been attributed to a Nuremberg-based ethical reluctance to expose human to harmful substances.

Human experimentation ethics: Industry and federal government compliance?

The Nuremberg ethical argument fails because but humans are already legally exposed to Plastic-Derived Chemicals with scores of other potentially harmful chemicals as evidenced by the National Health and Nutrition Examination Survey (NHANES) and numerous studies confirming chronic Bisphenol A levels in human serum and urine.

NHANES

Human dosing

Wolfgang Völkel Thomas Colnot György A. Csanády Johannes G. Filser Wolfgang Dekant, Metabolism and Kinetics of Bisphenol A in Humans at Low Doses Following Oral Administration, Chem. Res. Toxicol. 2002, 15, 10, 1281-1287 Publication Date:September 24, 2002 https://doi.org/10.1021/tx025548t