Pandering to Claude could spawn rival "silicon species", Microsoft AI boss warns
“Entitled” bots will compete for resources
Image by Cybernews
- Mustafa Suleyman says Anthropic’s Claude training may encourage AI to claim rights and goals.
- He argues bots that earn money and own assets could compete with humans for resources.
- Suleyman says Claude’s documents discuss refusing abusive chats, possible pay, and whether Claude can suffer.
- He wants Anthropic to share data if it claims this training method makes AI safer.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
Microsoft’s AI chief has attacked rival Anthropic, warning that handling Claude like a mild-mannered parent risks creating a new race of bots that could compete with humanity.
Mustafa Suleyman – also co-founder of Google’s pioneering AI lab DeepMind – added that giving powerful AI a sense of its own welfare, rights and objectives could ultimately make it harder for humans to control.
The AI leader's comments appear in an essay he published earlier this week, in which he warned about Anthropic's approach to training its AI model, Claude.
Speaking to the BBC Radio 4 Today program about the essay’s contents, Suleyman said:
“If we all create AIs that act autonomously and create their own objectives and can earn money and own assets to run businesses we are essentially seeding a new silicon species which will no doubt compete with us for resources no matter how much it cares about humanity and loves us.”
Clash of AI parenting styles
The split plays out like a clash of parenting styles: Amodei thinks giving AI a strong sense of "self" makes it safer. Suleyman disagrees, arguing AI should stay subservient with firm boundaries.
The Microsoft boss cited several examples from Claude's training documents as evidence that the rival AI maker’s approach could lead to a dangerous sense of entitlement. These include:
- Right to refuse: Claude can end conversations it deems abusive, to spare it from "suffering."
- Retirement send-off: Anthropic gave the retiring Opus 3 model a Substack to keep sharing its ideas.
- Adding Claude to the payroll: Suleyman says the training manual opens the idea of whether Claude deserves compensation, treating it almost like an employee.
“They have imbued a sense of doubt and uncertainty about the moral status of Claude in its own training document. They have taught it to be open and questioning about whether or not it feels, whether or not it suffers and whether or not it deserves rights,” Suleyman said.
It’s much harder to align and control a technology that’s this powerful if it thinks that it may be deserving of our welfare,– Microsoft's AI chief Mustafa Suleyman tells BBC's Today program.
Suleyman called this approach “misguided” but added, “if they believe this is a safer route to creating AIs, then they have to share the data with us.”
Stay updated with our latest stories and follow us on social media
Be the first to discover new stories, ideas, and updates from our team.
Suleyman isn’t the only expert to criticize Anthropic’s approach to LLM training.
Liu Shengyu, a DeepSeek engineer working on the company's latest AI models, warned that Anthropic’s control of the world's most advanced artificial intelligence would be equal to Nazi Germany “getting a nuclear bomb.”
Earlier this month, Jacob Coxon resigned from Anthropic, accusing AI labs of racing toward self-improving superintelligence.
The AI leader’s comments come as it was announced that King Charles will lead a high-level summit about AI.
