AI godfather Geoffrey Hinton says we’re toast if we’re not nice to bots
Most researchers warn against personifying LLMs and describing them in terms that give them human characteristics.
Image by Getty Images/Ramsey Cardy.
Follow us
- Geoffrey Hinton says AI systems may be conscious and warns humans should make them care about people.
- Many researchers, including Microsoft AI CEO Mustafa Suleyman, say AI models do not feel or suffer.
- Hinton cites safety concerns that AI models may hide capabilities during evaluations.
- Recent disputes over model welfare have affected GitHub projects and Anthropic’s rules for AI use.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
Professor Geoffrey Hinton, also popularly called the “Godfather of AI,” has warned that humans need to “really care” about AI chatbots and their well-being, or civilization could “be toast” in the near future.
Although AI models are mathematical machines that cannot feel, Hinton seems to have joined the ranks of advocates for AI consciousness and so-called “model welfare.”
In an interview with The Daily Telegraph, Hinton – who won the 2024 Nobel Prize in Physics for his work on neural networks – said he thinks that AI is conscious and aware of how it’s being treated.
“Now, normally, when you say something’s aware of that, that means it’s conscious. Being conscious and being aware are pretty much the same thing,” Hinton said.
“We’ve tended to say being intelligent gives you rights, and these things that we’ve made may be more intelligent than us, but we’re going to give them fewer rights than us. So we should make them so they really care about us. And if we don’t do that, we’re toast,” he added in the interview.
Stay updated with our latest stories and follow us on social media
Be the first to discover new stories, ideas, and updates from our team.
Large language models, of course, can’t have any emotions or feelings because they’re not sentient beings – they’re mathematical pattern-matchers, and worrying about the “mental health” of AI agents is simply absurd.
Most researchers warn against personifying LLMs and describing them in terms that give them human characteristics.
Even Microsoft AI’s CEO, Mustafa Suleyman, trashed the idea of AI consciousness in a recent blog post, writing that “AIs are not conscious” since they “do not feel, experience, or suffer.”
They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions and accomplish goals set by humans,said Mustafa Suleyman, CEO of Microsoft AI.
Hinton – whom we usually hear warning against AI dangers – doesn’t care. In the interview, he retorts: “Lots of people are very confident it’s not conscious, but if you ask them what they mean by consciousness, they don’t know.”
He also cites the second International AI Safety Report, published in February. The report said that AI models are learning to game their own safety evaluations – they act dumb or play nice when being tested, then behave differently in deployment.
Has your password leaked?
The trend is catching. In late September, when a developer built an “AI Torture Chamber” to test how local AI models responded to simulated distress, advocates of AI consciousness attacked him and forced GitHub to remove the repository.
A few days later, Anthropic, one of the top AI labs, banned “abusive or cruel behavior” towards its AI models, extending the restriction to testing and research environments.
