AI at play? White House health report cited non-existent studies

US Health Secretary Robert F. Kennedy Jr. has boasted that his Make America Healthy Again Commission report is almost flawless and based on “gold-standard” science. But the document is actually rife with errors and seems to have been at least partly generated by AI.

NOTUS, a news organization, took a closer look at the report released last week, and found that some of the cited studies didn’t actually exist at all.

These included fictitious studies on direct-to-consumer drug advertising, mental illness, and medications prescribed for children with asthma.

For example, Katherine Keyes, a professor of epidemiology at Columbia University, was listed as the author of a paper on mental health and substance use among adolescents. But she hasn’t actually written any paper by the title the report cited.

Moreover, Dr. Ivan Oransky – who teaches medical journalism at New York University and is a co-founder of Retraction Watch, a website tracking retractions of scientific research – told The New York Times that the errors in the report were characteristic of the use of generative AI.

Asked at a news conference whether the report had relied on AI, the White House press secretary Karline Leavitt spoke about “some formatting issues.”

After assuring that the report would be updated (it has been), Leavitt said: “The substance of the MAHA report remains the same – a historic and transformative assessment by the federal government to understand the chronic-disease epidemic afflicting our nation’s children.”

Although chatbots such as ChatGPT can facilitate cost-effective text generation and editing, factually incorrect responses – hallucinations – limit their utility in various areas, including citations in scholarly works, researchers have long said.

Of course, the false references don’t necessarily mean the statements and arguments in the report are inaccurate. But a lack of rigorous review and verification of the report and its bibliography before it was released throws doubt around the whole process.

The 72-page report calls for increased scrutiny of the childhood vaccine schedule and describes America’s children as overmedicated and undernourished.

Despite outspending peer nations by more than double per capita on healthcare, the US ranks last in life expectancy among high-income countries and suffers higher rates of obesity, heart disease, and diabetes.