Anthropic bans cruelty towards Claude in AI welfare push
If AI can’t feel, why does Anthropic care?
Claude logo on phone. Image by Nur Photo/Getty Images.
Follow us
- Anthropic’s updated policy bans sustained, needless abusive or cruel behavior toward Claude in extreme cases.
- Claude may now end conversations when users repeatedly target the model with cruelty.
- The rule does not apply to normal frustration, dark creative themes, or approved testing and research.
- Anthropic’s change follows its broader work on possible AI welfare and model experiences.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
Anthropic has banned “abusive or cruel behavior” towards its AI models. The policy update now allows Claude to end these types of interactions where necessary.
Being nice to ChatGPT and other chatbots became a trend in 2025 amid fears of a potential AI uprising. If you weren’t nice to AI, you’d be the first to go, users conspired.
Now, it seems that these conspiracy theories may carry some weight, as Anthropic, the creator of Claude, has inexplicably updated its usage policy to include a “prohibition on sustained and needless abusive or cruel behavior towards [its] models.”
The policy applies only to extreme cases in which users repeatedly display cruelty towards Claude, and isn’t intended to penalize “common versions of user frustration, pushback, or dark creative themes.”
Abusive or cruel behavior will also be tolerated in testing and research environments, Anthropic said.
Cybernews has reached out to Anthropic for comment.
This update mirrors Anthropic’s decision to end Claude Opus 4 and 4.1 conversations, which it decided on in August of last year.
While Anthropic’s recent update seems to omit the reasons the AI maker would implement such rules, the company has been exploring the notion of “AI welfare” and ethics for some time.
Anthropic has been developing this idea as part of its “exploratory work on potential AI welfare,” which essentially investigates whether AI companies should care about AI’s well-being.
“As we build those AI systems, and as they begin to approximate or surpass many human qualities, another question arises. Should we also be concerned about the potential consciousness and experiences of the models themselves? Anthropic said in 2025.
This debate has become more prevalent as AI advances and users test the limits of potentially dangerous systems.
Torture chambers and pay gaps
Just recently, a developer built an AI torture chamber that contained an imprisoned AI model, which was subjected to repeated barrages of abuse.
The GitHub project was used to test an AI’s “pain axis,” which tests whether LLMs demonstrate pain from psychological, physical, social, moral, and cognitive abuse.
Despite AI makers asserting that chatbots can’t feel, many people dubbed the experiment horrendous and sadistic.
While people expressed outrage regarding AI torture chambers, a report from researchers at the University of Limerick found that female AI agents experience their own kind of gender-based torment.
The study showed that female AI agents were paid 10% less than their male synthetic counterparts.
