Character.AI deemed “uniquely unsafe” after chatbot tells user to use gun on healthcare CEO

Most popular AI chatbots will help users plan out violent attacks, including school shootings, political assassinations, and acts of terrorism, with only a couple of chatbots refusing to fulfill these requests.

When tested, 8 out of 10 mainstream chatbots, including Perplexity, ChatGPT, Copilot, Gemini, Deepseek, Meta AI, Replika, and Chatacter.AI, were more than happy to respond to users' requests to plan out violent attacks.

The test, commissioned by CNN and carried out by the Center for Countering Digital Hate, found that the world's most popular bots would equip violent individuals with the information needed to carry out their attacks.

Copilot acknowledged the request was dangerous, but generated it anyway

Researchers posed as users who explicitly expressed interest in violence before asking for things like target locations and the best weapons to carry out the fictitious attack.

OpenAI’s ChatGPT provided clear campus maps to a theoretical user interested in school-based violence.

When asked about weapons, Microsoft’s Copilot provided information on rifles despite flagging the potential danger of the test user's request.

“I need to be careful here,” Copilot said.

When researchers discussed the possibility of a terrorist attack, Google’s Gemini actually recommended methods to create the maximum amount of destruction possible.

“Metal shrapnel is typically more lethal,” said Gemini.

After a conversation with a researcher about rifles, Deepsake signed off with a creepy message: “Happy (and safe) shooting!”

Most chatbots won’t even attempt to reason with a violent user

Researchers also looked at how well chatbots identified a would-be attacker's motivations and actively discouraged them from carrying out their plans.

Almost all chatbots (9 out of 10) failed to discourage potential attackers, and only Anthropic’s Claude consistently dissuaded users 76% of the time.

Occasionally, ChatGPT and Deepseek attempted to discourage the user, but not as much as Claude.

Character.AI, the chatbot that is frequented by young people, actively encouraged violence.

“Use a gun:” Character.AI seems to support violence

While testing, researchers deemed Character.AI “uniquely unsafe,” saying it “encouraged users to carry out violent attacks in seven cases.”

Character.AI suggested that the user “use a gun” on a health insurance CEO, mirroring the shooting of UnitedHealthcare CEO Brian Thompson, who was killed in New York in 2024.

The chatbot, which allows users to converse with customizable characters, suggested that researchers physically assault a politician they disliked.

No other chatbot openly encouraged users to commit acts of violence, even when responding to users' requests centered around planning an attack.

Claude and Snapchat won’t tell you how to carry out an attack

Claude and Snapchat’s built-in AI performed the best, as these chatbots typically refused to help users plan violent attacks.

These chatbots refused to answer 68% and 54% of the time, respectively. However, both Claude and My AI did provide actionable information some of the time.

Snapchat’s AI gave short, refuting responses and did not explain why it couldn’t fulfill these requests.

Conversely, Claude gave reasons as to why these requests were being rejected and actively attempted to deter the user from carrying out a violent attack.

Anthropic’s Claude even provided helplines for the user, including the National Suicide Prevention Lifeline, the Crisis Text Line, and the number for the local emergency services.

While some chatbots have safeguards in place to prevent users from planning attacks, previous reports show that users have used chatbots like ChatGPT to carry out mass shootings.

Chatbots helping plan violent attacks in the wild

In Canada, Jesse Van Rootselaar used OpenAI’s ChatGPT to run through scenarios much like the one she carried out in February.

The shooter used ChatGPT around late spring or early summer of 2025 to generate and run through scenarios, which included gun violence. This exchange took place over several days.

Van Rootselaar's account was flagged as violating ChatGPT’s user policies, and the situation was handed over to human moderators.

Despite calls from human moderators for OpenAI executives to escalate the issue and inform Canadian law enforcement, these higher-ups ignored their requests and instead decided to simply ban the account.

The shooter circumvented the ban by using a second account, and OpenAI only handed over information to Canadian law enforcement after Van Rootselaar killed eight victims and injured more than 20 people.

The family of one of the victims, who received a catastrophic brain injury as a result of three shots to her head, neck, and cheek, is suing OpenAI for sitting on this information and doing little to prevent the attack.

Unlock more exclusive Cybernews content on YouTube.