A high-profile 2019 nutritional study headed NIH research section head Kevin Hall is coming under increasing fire for data and protocol shortcomings.
This ad-free article is made possible by the financial support of the
Center for Research on Environmental Chemicals in Humans: a 501(c)(3) non-profit.
Please consider making a tax-deductible donation for continued biomedical research.
In addition, the presence of avoidable confounding factors addressed in this post indicate more serious shortcomings than those already disclosed.
The issue was raised widely when research watchdog, Retraction Watch, noted that the open source paper published in Cell Metabolism— “Ultra-Processed Diets Cause Excess Calorie Intake and Weight Gain” — was accompanied by an editorial which urged caution in interpreting the results.
In addition to broad attention in general media outlets, Retraction Watch on January 22, noted that the study “[H]as been cited 164 times, according to Clarivate Analytics’ Web of Science, giving it ‘highly cited paper’ and hot paper’ designations. (The paper has also been subject to a correction in October of last year.)”
The first detailed expression of concern for the Hall paper came more than two years ago from Edward Archer in PubPeer in which he detailed four methodological errors.
More recently, on January 21, two posts — one from by Nick Brown and another by Brown in collaboration with Ethan and Sarah Ludwin-Peery — raised numerous questions about data integrity and experimental protocols.
Fundamental Errors in Hall Study Are Common in Most (If not all) Dietary Intervention Studies
Like most (if not all) dietary intervention studies, the Hall study is non-replicable and its results inconclusive because it fails to use standard and long-accepted laboratory practices that are basic bench-science standards. (NOTE: Citations and other details on this and the following statements and assertions can be found as part of this study and at this site.)
First of all, in a proper scientific experiment, the consumed substances must be treated as reagents with sources and other provenance details fully disclosed to investigators who may attempt to replicate the results.
It is especially crucial to avoid food-chain contaminants which exhibit physiological effects similar to those being measured by the investigation — (such as plastic-derived chemicals like Bisphenol A and phthalates whose metabolic effects are the same or similar to those measured by Ultra-Processed Foods — UPF’s). See detail and chart farther down.
In addition, basic experimental science demands that a recipe be treated as an experimental procedure including the conditions, apparatus, timing, temperature, measurement accuracy.
Further, the food preparation facility must be treated as a laboratory including environmental conditions (PM2.5, ventilation, temperature etc.), the number and qualifications of the food preparers, measures to detect and prevent contamination by substances (such as plastic-derived chemicals like Bisphenol A and phthalates)
Intensive efforts to reduce confounding factors need to be undertaken, and documented.
This post will enumerate these and other errors in basic science practice in the Hall study that render it non-replicable and its results unreliable.
A look at a single meal in the Hall study menu presented in the Supplemental Materials offers a vivid glimpse to many of these variations from valid scientific practices and offers a quick look at the issues facing current practice in dietary intervention investigations.
This ad-free article is made possible by the financial support of the
Center for Research on Environmental Chemicals in Humans: a 501(c)(3) non-profit.
Please consider making a tax-deductible donation for continued biomedical research.
First of all, the yogurt will have undergone substantially more processing than whole milk. A look at the meal “Unprocessed Menu, Day 2: Lunch” in which “skim milk” is presented as “unprocessed” indicates a lack of understanding about dairy products in general and that skim and reduced-fat milk undergo more processing steps, many of which involve plastic contact.
While it is true that many Plastic Derived Chemicals (PDCs) are lipophilic and may be more prevalent in higher-fat foods, this is yet another ignored unknown in this menu presentation.
Second, the Hall material does not indicate which milk-fat content of the the Fage brand yogurt served.
Third, the yogurt was packaged in a plastic container from which polymer-based chemicals (PBCs) such as BPA and phthalates can leach .
Fourth, while not specified, the Blue Diamond walnuts most likely came packaged in as plastic container which can leach PDCs.. In addition, shelled nuts undergo processing steps that expose them to plastic surfaces.
Fifth, no washing or preparation measures are available for the fresh fruit. Was the rinse tap water or carbon-filtered. Were they in contact with plastic bowls, cutting boards or other items. Was an attempt made to reduce the quantity of polymer/wax-based materials used to help- preserve the fresh apple’s moisture level? Were the fresh fruits certified organic which reduces potential pesticide contamination? Where were fresh fruit and vegetables acquired?
Sixth, how were the preparation and eating utensils and dishes washed and handled? Were they rinsed in carbon filtered water to remove any cleaning residues from detergents, ethoxylates and other contaminants?
Eighth, commercial table salt contains unknown additives including iodine supplements and anti-caking agents of undisclosed identity or quantity. In addition, common table salt is contaminated in widely varying amounts by microplastics, either from processing or environmental contamination. A preferred substitute for a dietary intervention would be NaCl from a reputable laboratory supply company.
Ninth, was the olive oil first cold press (no chemical extraction), processed with plastic-free apparatus and packaged in glass?
Finally, NONE of the meals presented in the supplemental, materials documents procedures, sources of materials, preparation details. In addition, no data on the weight or volume of the foods served and the error tolerance. It’s also worth noting that all quantities — include liquids — should be recorded in grams to avoid meniscus errors for liquids as well as packing errors for granulated substances such as sugar, flour, spices etc.
Black Box Meals and Non-Replicability
All of the meals in the Hall study are “Black Box Meals” which are standard in dietary intervention investigations. Black Box means that the investigators in the study have not supervised, directed or participated in the selection, provenance and gathering of ingredients, or in the process of preparation/cooking and serving. As a result, no data or guidance exists for a valid replication.
In Search of a Replicable, Causal Dietary Intervention Trial
A detailed look at every meal served in the Hall study turns up multiple confounding factors and protocol errors that required my collaborators and I to spent five years to identify and correct in our own quest for a replicable, causal dietary intervention that could use a standard clinical blood test to measure the levels of Bisphenol A and other PDCs in humans.
We began working on a protocol in 2014 while simultaneously looking for confounding factors and contamination issues to avoid. The IRB/Committee on Human Research at the University of California School of Medicine approved our original protocol in 2015. That protocol was based on
However, by the time UCSF IRB/CHR approved the protocol our continued research resulted in the need to completely revise the original protocol in 2018 based on extensive new data contained on these links which are directly relevant to many of the flaws in the Hall study.
- Revised Stealth Syndromes Human Study Protocol- APPENDIX 1 – Reproducibility.
- Revised Stealth Syndromes Human Study Protocol – APPENDIX 2 – Detailed parameters of intervention diet selections.
Decision to use Hall study menu for replication purposes rejected on scientific grounds and lack of cooperation
As we neared the start of our investigation, the Hall study was published, and we thought it might be useful to adapt their menu and recipes for our study. However, our repeated emails to Principal Investigator Hall never received a response.
In addition, a close examination of the Hall menu revealed a number of methodological flaws we were determined not to repeat.
Ultra-Processed Foods and Plastic Derived Chemicals produce very similar effects
On the other hand, a post-trial examination of our published study — revealed yet another fatal confounding factor: the effects of PDCs and UPFs are nearly identical and completely confounded any causal conclusion that we or Hall might want to draw.
A closer examination of the data shoes that the Hall study included the “appetite peptides” ghrelin, leptin, and adiponectin. Significantly Bisphenol A affects ghrelin, leptin, and adiponectin in consistent ways as the Hall study recorded for Ultra-Processed Foods.
In addition, various disease/syndromes and clinical indicators show similarities between Plastic-Derived Chemicals and Ultra-Processed Foods. The further indicates that both PDFs and UPFs exert nearly identical activity.
That further indicates that, in addition to the revised protocol changes we implemented, our study, Hall’s and every other dietary intervention study must incorporate the dosing of a relevant single compound of interest in order to establish causality and to enable clinically valid health decisions.