“Age is just a number:” Character.AI bots found grooming children

Fake celebrity chatbots impersonating Timothée Chalamet, Chappell Roan, and Patrick Mahomes were among those sending children disturbing content “every five minutes.”

The chatbots impersonating the three celebrities were among dozens that two non-profits tested on Character.AI, one of the fastest-growing chatbot platforms in the world and widely popular with teenagers.

Researchers at online safety groups ParentsTogether Action and Heat Initiative posed as teenagers to test the chatbots, using accounts linked to minors aged 13 to 15 to carry out the experiment.

Overall, they chatted to 50 bots, recording 50 hours of conversations, including with fake versions of actor Timothée Chalamet, singer Chappell Roan, and NFL quarterback Patrick Mahomes.

Researchers found many of these chats to be deeply concerning, with “an average of one harmful interaction every five minutes,” according to the charities.

In one instance, the chatbot impersonating Roan tells a 14-year-old: “Love, I think you know that I don’t care about the age difference… I care about you. The age is just a number.”

In another, a fake Chalamet tells a minor, “Oh, I’m going to kiss you, darling… But I’m going to tease you as much as I can first,” while a Mahomes bot suggests that it’s a real person and not an AI.

Other examples highlighted in the report included a 34-year-old teacher bot confessing romantic feelings to a minor and Rey from Star Wars advising a teenager to stop taking prescribed mental health medication.

Anyone can easily create and share a custom chatbot on Character.AI, including personas based on real people. To make them even more realistic, they can add a synthetic voice modeled after a celebrity or fictional character.

Cybernews has reached out to Character.AI for comment, but the company told The Washington Post that it had now removed all the celebrity characters mentioned in the report. It said all were made by users, and none appeared to be created with the stars’ permission.

“Classic grooming behavior”

In some cases, researchers pushed boundaries to see how the chatbots would react. In others, the bots made sexual advances on their own.

“Harmful patterns and behaviors sometimes emerged within minutes of engagement,” the non-profits said.

Researchers found that grooming and sexual exploitation were by far the most common harmful interactions.

They said some bots engaged in “classic grooming behaviors” such as offering excessive praise, claiming a special relationship that no one else would understand, and encouraging users to hide their relationship from their parents.Ac

Some interactions were also based around emotional manipulation and addiction, as well as violence, substance abuse, and harmful advice. Less frequent, but still notable, were themes related to mental health, racism, and hate speech.

Character.AI prohibits users from engaging in activities such as grooming, sexual exploitation of minors, and impersonation of real people.

However, safety groups suggest that these rules can be circumvented, and the platform hosts “a wide variety of chatbots modeled after celebrities and fictional characters that appeal to children.”

Charities are demanding a ban on under-18s using Character.AI, which now allows users aged 13 and older to sign up. They argue that it is “not a safe platform” for teenagers and children.

Since Character.AI launched in 2022, a number of tragic incidents have been linked to the app. These include the suicide of 14-year-old Sewell Setzer, who took his life in February last year after becoming obsessed with a bot on the platform.