Anthropic CEO joins AI doomer ranks with rambling essay, but what are his true motives?
A lot of what Anthropic cofounder and CEO Dario Amodei writes about AI risks in his new 19,000-word essay is spot-on. But the general idea that AI can doom us all to a grim future? Let’s just say it’s the dark side of the whole hype.
In the essay posted to his blog, Amodei argues that “humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.”
Sounds familiar, doesn’t it? That’s because this is the sort of language the world’s most famous AI doomers, such as OpenAI’s CEO Sam Altman, Yoshua Bengio, or Geoffrey Hinton, use.
Are the risks actually overblown?
“Humanity needs to wake up, and this essay is an attempt – a possibly futile one, but it’s worth trying – to jolt people awake,” Amodei goes on.
The Anthropic CEO then states that we – humanity – are closer to real danger in 2026 than we were in 2023. Indeed, jobs are being lost (Amazon is particularly ruthless), and economic power is further being concentrated in the hands of the few broligarchs.
But that’s not what Amodei means, mind you. In the essay, he also writes about the possibility of AIs developing dangerous bioweapons or “superior” military weapons, and contemplates the idea that an AI could go rogue and, naturally, “overpower humanity.”
Now, some skeptics say that the existential AI risks might be overblown or simply elsewhere, and others claim that AI doomerism is just part of the same AI hype.
In their brilliant book “The AI Con,” Emily Bender and Alex Hanna point out: “The tactic seems to be to take a series of tropes from science fiction (they’re all nerds) about how some Terminators would wage wars for their right to exist, and then actually make everyone talk about it.”
“But this belies what these technologies are doing to the rest of us: threatening stable careers and replacing them with gig work, slashing personnel in government, cheapening our social services, and degrading creativity,” write Bender and Hanna.
“If you really think there’s existential risk, why are you working on this at all? That's a pretty fair question to ask,”
Gary Marcus.
In 2023, Gary Marcus, a New York University emeritus professor, refused to sign an open letter calling on global leaders to reduce “the risk of extinction” posed by AI technology and told AFP that he wasn’t personally concerned about it.
“If you really think there’s existential risk, why are you working on this at all? That's a pretty fair question to ask,” Marcus said.
Cards on the table
To be fair, Amodei does well when, right at the beginning of the essay, he advises us all to “avoid doomerism,” saying that during what he calls the peak of worries about AI risk in 2023-2024, “some of the least sensible voices rose to the top.”
The Anthropic CEO also smartly – and rightly – takes a dig at Elon Musk’s Grok chatbot: “Some AI companies have shown a disturbing negligence towards the sexualization of children in today’s models, which makes me doubt that they’ll show either the inclination or the ability to address autonomy risks in future models.”
But then Amodei puts his cards on the table, perhaps revealing the real motivation behind the essay. He writes: “AI is so powerful, such a glittering prize, that it is very difficult for human civilization to impose any restraints on it at all.”
One could remind him of regulation, of course, but AI companies, including Anthropic, don’t want to hear about it. Besides, Amodei seems to think that, on the contrary, less carefulness could help democracies thrive and shoo away tyranny.
The conversation on this topic is live. Join in the discussion.
“Taking time to carefully build AI systems so they do not autonomously threaten humanity is in genuine tension with the need for democratic nations to stay ahead of authoritarian nations and not be subjugated by them,” Amodei says.
“AI-driven terrorism could kill millions through the misuse of biology, but an overreaction to this risk could lead us down the road to an autocratic surveillance state.”
For good measure, Amodei also urges the US to deny other countries the resources to build powerful AI systems and says that America selling Nvidia AI chips to China is equivalent to “selling nuclear weapons to North Korea and then bragging that the missile casings are made by Boeing and so the US is ‘winning.’”
It all sounds very noble indeed. But in the AI industry, nobody is really altruistic: billions of dollars are at stake, at least until the bubble bursts.
According to The Wall Street Journal, Anthropic is looking to close a massive round of funding at a valuation of $350 billion. Quite obviously (although paradoxically), Amodei is keen to position his company as the solution to the risks he – and the entire AI community – had a hand in bringing about.
Unlock exclusive Cybernews content on YouTube.
