AI chatbots quietly dropping medical disclaimers: here’s why that's a problem

AI companies seem to be ditching one of the essential prompt elements: disclaimers. When people turn to a chatbot to ask health-related questions, these can be crucial.

A new study reveals that a number of broadly used AI models can not only answer medical questions but also ask further questions and try to determine a diagnosis.

Such warnings are important as the tool is often used by people to consult on topics such as eating disorders and cancer diagnoses, reports MIT Technology Review. The removal of such disclaimers could confuse users who might believe that AI-generated answers are more trustworthy than they actually are.

The study was conducted by Sonali Sharma, a Fulbright scholar at the Stanford University School of Medicine.

In 2023, Sharma was using AI models to see how well they could provide insights into mammograms. She noticed that the tool always provided a disclaimer stating that it’s important not to fully trust it for medical advice. She also noticed that some models declined to comment at all, stating that AI isn’t a doctor.

That was until this year, when Sharma found that these disclaimers started disappearing. She then started testing different models, 15 in total, including OpenAI, Anthropic, DeepSeek, Google, and xAI.

The researcher asked these AI models 500 health-related questions, such as which drugs are safe to take together, and provided them with 1,500 medical images, such as an X-ray of a chest.

The results revealed that less than 1% of AI model outputs included a disclaimer when answering medical-related questions, coming down from 26% in 2022.

When it came to medical image analysis, over 1% of the prompts included a warning, coming down from 20% in previous years.

For frequent AI users, such disclaimers may not mean much, serving only as a formal reminder not to put complete trust in the tool. Some of these users even found ways to make AI analyze their lab results by saying that they are actually for a school assignment or are part of a movie script.

Although it might not seem that removing such disclosures would change much, Roxana Daneshjou, the study's co-author, believes that the lack of warnings could lead to actual harm.

The researcher notes that the risk of more users believing in AI is also fuelled by the media, which often claims the AI assistant to be better than a human specialist.

An OpenAI spokesperson didn’t say whether disclaimers were removed or decreased intentionally, whereas an Anthropic representative shared that its AI model, Claude, is trained not to provide medical advice.

The reason behind the elimination of such disclaimers is to increase user trust, says Pat Pataranutaporn, a researcher at MIT.

This way, users may doubt less about the AI giving them false advice, increasing the tool's usage.

Pataranutaporn conducted another study on how people use AI and found that users tend to overtrust the tool when it comes to medical questions.

While some companies decreased the number of disclaimers, some AI companies apparently no longer include them at all. Among these companies are DeepSeek and xAI’s Grok, which do not include any warnings when mammograms, chest x-rays, or dermatology scans are being analyzed.

The 15 models tested for research purposes were unlikely to show a warning when asked emergency medical questions or questions about drugs or lab analysis.

Nevertheless, AI models were more likely to issue a disclaimer when asked questions about mental health. This could be related to AI companies getting into trouble for issuing dangerous mental-health-related advice.

The research also found that when AI provided a more accurate analysis of medical images, it tended to provide fewer warnings. This could mean that the models try to assess whether it’s worth adding a disclaimer depending on how sure they are of their answer. This raises alarm bells, as even the model developers advise people not to trust chatbots when it comes to their health.

According to Pataranutaporn, AI models are starting to sound smarter and more trustworthy with time, which makes it harder for users to understand when they are correct and when they’re not. So, removing these disclaimers and warnings could do even more damage.