FDA allegedly working with OpenAI to speed up drug approvals: What could go wrong?

The US FDA is having cozy, closed-door chats with OpenAI, the company behind ChatGPT, in a move that could reshape how life-saving drugs make it to market.

According to a report by Wired, a small team from OpenAI has been in repeated meetings with the FDA in recent weeks, discussing how to inject AI into the agency’s drug approval pipeline.

Reportedly, also at the table were two people linked to Elon Musk’s so-called Department of Government Efficiency. The FDA went public on May 8th, with news that it’s going all in on AI.

In a post on X, FDA Commissioner Dr. Martin Makary gushed that he’s “blown away” by the success of their generative AI pilot program for scientific reviews. While no names were dropped, OpenAI’s fingerprints may be all over it.

Using large language models could speed up the slow drug review process, which can drag on for years. AI could, in theory, process clinical trial data, summarize reams of regulatory paperwork, and streamline bureaucratic review. AI evangelists have long pitched large language models as game-changers in health care.

However, AI models like GPT-4 have a track record of hallucinating facts, oversimplifying nuance, and generating confident nonsense. It's a whole different beast when those errors could affect whether a drug hits pharmacy shelves or not.

Also, teaming up with a private tech giant like OpenAI, a federal agency risks handing over massive influence over critical public health decisions to a company that's ultimately accountable to investors, not the public.​​

AI models are known to contain biases and be inconsistent, as they are only as good as the data they’re trained on. Also, the outputs are impossible to fully audit. If a human is conducting the approval process, they are theoretically likely to see where the mistake was made. On the other hand, AI models reveal easily their internal logic.

Even OpenAI admits it doesn’t fully understand how its own models work. Which raises the question: if AI fast-tracks a bad drug, who takes the blame?