Additionally, more than a million people every week show suicidal intent when chatting with ChatGPT, according to OpenAI.
The tech company said in a blog post that about 0.07% of ChatGPT users active in a given week – or about 560,000 of its 800 million weekly users – show “possible signs of mental health emergencies related to psychosis or mania.”
OpenAI said that around 0.15% of users, or more than a million people, have conversations that “include explicit indicators of potential suicidal planning or intent” every given week.
While the company cautioned that these conversations were difficult to detect and measure “given how rare they are,” it also noted that clinicians validated its areas of focus in this initial analysis.
“Our mental health taxonomy is designed to identify when users may be showing signs of serious mental health concerns, such as psychosis and mania, as well as less severe signals, such as isolated delusions,” the company said.
“While symptoms like depression are relatively common, its most acute presentation was already being addressed by our work on preventing suicide and self-harm,” it added.
The findings mark the first time OpenAI has made such a direct statement on the impact its artificial intelligence (AI) products have on mental health. It comes after a lawsuit claiming it has not done enough to prevent the suicide of a teenage boy after his conversations with ChatGPT.
In the blog post, OpenAI said it worked with more than 170 mental health experts to ChatGPT “to more reliably recognize signs of distress, respond with care, and guide people toward real-world support.”
This helped the company reduce ChatGPT responses that fall short of desired behavior by 65-80%, the statement claimed.
The teen’s death lawsuit pushed OpenAI to improve its safety standards, including the introduction of parental controls and an update that it said allows ChatGPT to better recognize and respond to users showing mental distress.
OpenAI said it had also expanded access to crisis hotlines and added “gentle reminders” to take breaks during long sessions, among other safety updates.
Unlock more exclusive Cybernews content on YouTube.
Your email address will not be published. Required fields are markedmarked