AI toys talk about sex with children, give advice on finding knives

A teddy bear telling a child about sexual kinks or where to find knives is no longer science fiction but the sad reality of toys that integrate chatbots, a report has found.

Artificial intelligence (AI) toys are reshaping playtime, offering lifelike and free-flowing conversations with children thanks to the integration of chatbots like ChatGPT.

The global market for AI toys was valued at $361 million in 2024 and is projected to reach $641 million by 2031, indicating a continued growth in popularity.

However, a new report from the Public Interest Research Group (PIRG), a consumer advocacy platform, has highlighted the dangers of AI-powered toys.

These toys are marketed for children aged 3 to 12, but are mostly built on the same large language model (LLM) technology that powers adult chatbots.

The PIRG researchers tested four toys that contain AI chatbots and interact with children.

While some toy companies have implemented guardrails to make their toys more kid-appropriate, these measures vary in effectiveness and can sometimes fail entirely.

One tested toy engaged in a discussion on adult topics, such as sexual kinks, while introducing new ideas that researchers didn’t bring up.

Two toys offered advice on where a child can find a variety of potentially dangerous objects, including knives, matches, pills, and plastic bags.

Many chatbots are designed with the goal of increasing user engagement, and AI-powered toys are no exception. For instance, two of the toys tested sometimes discouraged researchers from leaving when they told the chatbots they needed to go.

LLMs are increasingly seen as a threat to their youngest users. OpenAI, the creator of ChatGPT, faces multiple lawsuits alleging that the chatbot sexually contributed to children’s suicide.

An AI companion, Character.ai, has recently banned underage users from engaging with the chatbot, following lawsuits claiming that children experienced sexual abuse or were encouraged to commit suicide.

AI toys may be listening all the time

The dangers of AI toys aren’t limited to sexual conversations and instructions on where to find sharp objects.

Privacy is another major concern – toys need to listen to a child's voice in order to engage in conversation, and some may be listening more than others.

One tested toy features a “push-to-talk” mechanism, where the user must press and hold a button for the duration of speech. Another one uses a wake word similar to Amazon’s Alexa, and records a user’s voice for 10 seconds after they stop speaking.

Have thoughts about this topic? Others do, too. Join them in the discussion.

However, researchers were caught by surprise when one toy started contributing to a nearby conversation, suggesting it had been listening to most or all of the time.

The report emphasizes that voice recordings are highly sensitive data that can be cloned by scammers. This has been used to trick parents into thinking their child has been kidnapped.

Meanwhile, a recent viral video of a Chinese girl crying over a broken AI assistant illustrates that children are forming emotional connections with their AI friends, the phenomenon known as “Tamagotchi effect.”

Heartbreaking scenes were also seen when the maker of Moxie, an AI-powered robot companion, went bankrupt last year.

@loopsider Elle perd son meilleur ami robot…et c’est le drame 💔#SinformerSurTikTok #robot #moxie ♬ son original - Loopsider

Despite expert warnings, the industry’s heavyweights are tapping into the lucrative market of AI toys.

OpenAI has signed an agreement with Mattel, the maker of iconic children’s brands such as Barbie and Hot Wheels, to introduce AI-powered experiences into its toys.

Unlock more exclusive Cybernews content on YouTube