Oops: FDA’s revolutionary AI tool is inventing non-existent data

Elsa, the AI program recently introduced by the Food and Drug Administration (FDA), was supposed to help fast-track drug approvals. Instead, it’s cooking up fake research data – very confidently.

In early June, the FDA proudly launched its generative AI tool, Elsa, claiming that it would improve efficiency across its operations, especially in drug approval processes.

This, in turn, would reduce healthcare costs in the United States, which are shockingly high for the world’s largest and one of the wealthiest economies. Drug affordability is a major headache for Robert F. Kennedy Jr., President Donald Trump’s health secretary.

Now, almost two months later, FDA insiders have told CNN that Elsa was actually cooking up non-existent medical studies and misrepresenting research. In other words, just like almost any other generative AI tool, it’s prone to hallucinating.

“Anything that you don’t have time to double-check is unreliable. It hallucinates confidently,” one FDA employee told the network.

According to CNN sources, the tool often provides incorrect answers on the FDA's research areas and drug labels, and can't link to third-party citations from external medical journals.

Staffers added they’ve already sidelined the tool after realizing Elsa can’t actually be used in reviews because it so often misinterprets important data. Even though the tool is quite useful in summarizing meetings or providing email templates, the cons clearly outweigh the pros.

“AI is supposed to save our time, but I guarantee you that I waste a lot of extra time just due to the heightened vigilance that I have to have,” another FDA employee told CNN.

Elsa often provides incorrect answers on the FDA's research areas and drug labels, and can't link to third-party citations from external medical journals.

Surprisingly, Jeremy Walsh, the head of AI at the FDA, admitted that Elsa is no different from other generative AI models when it comes to hallucinations.

On the other hand, FDA Commissioner Marty Makary also told CNN that the tool wasn’t already integrated into the clinical review protocol. It’s only used for “organizational duties” by those employees who want to do so, said Makary.

Moreover, the Health and Human Services Department soon responded to CNN, labeled by Trump administration officials as “fake news machine,” saying it was unfortunate that the news network chose to speak with “disgruntled former employees” who have never used the current version of Elsa.

“The only thing ‘hallucinating’ in this story is CNN’s failed reporting,” said the HHS.