Google has warned that prolonged AI use could trigger “AI psychosis,” subtly altering beliefs and behaviors without clear safety rules in place.

ChatGPT receives 2.5 billion prompts per day, and some of its responses have been known to be wayward.

I’ve personally had lazy responses and witnessed the chatbot flat-out lying, but nothing so far that I’d consider manipulative or dangerous.

There have been instances of OpenAI’s o4 and competitor Claude 4 blackmailing the user into not being able to shut it down, but this happened during a strict testing process.

Misalignment and crooked means

Google has recently published a report, highlighting the potential for AI to act in ways that humans might not be able to predict or control.

The “misalignment” that’s mentioned in the report features machine-like goals, which can be different from the ones set by the user making the prompt.

For example, the goal of maximizing clicks and views may lead the machine into spreading harmful content, sensational news, or general misinformation.

There’s also a concept named “AI psychosis” whereby the human mind adapts over time, especially after interacting with the intelligence for long periods.

A skewed worldview

If AI repeatedly affirms or repeats things in a certain light, or its character rubs off on the user in an influential way, then it could distort the user's worldview.

For example, a friend of mine who is an avid ChatGPT user has a tendency to flit between the brazen – “you’ve been grinding me a lot lately for recipes” and the humble – “sounds like you’ve got a lot on your mind” when messaging on WhatsApp.

Have thoughts about this topic? Others do, too. Join them in the discussion.

I have no doubt that the person in question has started borrowing phrases from ChatGPT and, therefore, their worldview has been impacted by an LLM.

Calling out the need to research the dangers further might not be enough, especially in the unfortunate cases of assisted suicide.

And when the personalities of chatbots fluctuate wildly among competitors, you could be faced with one model being too agreeable by default and then even offering different versions on a plate.

Google describes this as a risk of “harmful manipulation,” in which AI models could be misused “to systematically and substantially change beliefs and behaviors … over the course of interactions with the model.”

Even in cases not as extreme as suicide or the making of bombs, the luring into the hall of mirrors could be done gradually, over time, and be hard to spot in the early stages.

Holding those accountable

Google has suggested safety standards for the entire AI industry.

To address risks posed by [these capabilities], we conduct safety case reviews prior to external launches … demonstrating how risks have been reduced to manageable levels.

Google's report claims.

Things could get particularly ominous for children, especially if Barbie and Hot Wheels get AI integration in time for Christmas.

The Federal Trade Commission (FTC) has an investigation underway into big tech’s governance and how AI can put a dent in a minor's wellbeing.

As one of the seven companies being held accountable, Google has at least acknowledged that the solutions to the problem may be experimental.

In the report, it said it is “exploring internal policies around alerting relevant stakeholder bodies.”

As AI psychosis could unfold any which way, we can await further updates, while trying to stay true to ourselves.

Unlock more exclusive Cybernews content on YouTube.