Scientist Gary Marcus says that applying hallucinating artificial intelligence (AI) to weapon systems without humans in the loop could be “catastrophic,” as Anthropic refuses to accede to the Pentagon’s demands.
This article is part of Cybernews’ weekly series, “AI week in quotes,” summarizing the most important developments in AI by quotes from the industry leaders, independent experts, and decision makers.
“Catastrophic” consequences of AI weapons
All eyes were on Anthropic this week, as the Department of War (DoW) issued the company an ultimatum to relax its safeguards or be blacklisted.
Anthropic previously said it objected to its AI tools being used for “mass domestic surveillance” and “fully autonomous weapons.” However, its tool Claude was used in the US military operation to capture ex-Venezuelan leader, Nicolas Maduro.
The company reiterated its position in a statement released on February 26th, citing its CEO, Dario Amodei, who expressed hope that the DoW would reconsider its position.
In a narrow set of cases, we believe AI can undermine, rather than defend, democratic values. Some uses are also simply outside the bounds of what today’s technology can safely and reliably do.
Dario Amodei
Gary Marcus, a scientist and best-selling author, wrote in the Bulletin of the Atomic Scientists that Anthropic’s showdown with the DoW may mean “life or death” for everyone.
Marcus said he wasn’t convinced that the Secretary of War, Pete Hegseth, has “a nuanced understanding of the strengths and limits of current AI.”
It should be obvious that applying hallucinatory and unreliable generative AI to weapon systems without humans in the loop could be catastrophic.
Gary Marcus
Rutger Bregman, a prominent Dutch historian and thinker, called Anthropic’s refusal to bow to the Pentagon’s pressure a “huge opportunity for Europe,” saying the continent should welcome the American AI company “with open arms.”
Europe already controls the AI hardware bottleneck through ASML. Add the world’s leading AI safety lab, and you have the foundations of an AI superpower.
Rutger Bregman
“Rookie mistake” with OpenClaw goes viral
Summer Yue, Meta AI’s director of alignment, stirred a heated discussion about the safety of autonomous agents after sharing how OpenClaw has nearly wiped out her inbox and ignored her instructions to confirm before taking any action.
Yue described the incident as a “rookie mistake” and said she got overconfident because the workflow had been working in her toy inbox for weeks.
The case was widely discussed in the media and on social networks. Economist and researcher Lucia Velasco noted that the incident happened to a person who is “one of the most qualified people in the world to handle it.”
The distance between how agents behave in the lab and how they actually do in the real world is still enormous. Being pro-adoption doesn't mean pretending autonomous agents are ready for everyone with no guardrails.
Lucia Velasco
AI makes programming “unrecognizable”
Andrej Karpathy, a leading AI researcher and co-founder of OpenAI, wrote that programming is becoming unrecognizable due to AI, with the major changes happening since last December.
In his post on X, Karpathy described how an AI agent took 30 minutes to complete the task, which would have been a weekend project three months ago.
You’re not typing computer code into an editor like the way things were since computers were invented; that era is over. You're spinning up AI agents, giving them tasks *in English* and managing and reviewing their work in parallel.
Andrej Karpathy
Just weeks ago, music streaming platform Spotify said its best developers “have not written a single line of code since December,” thanks to AI tools like Claude Code.
Unlock more exclusive Cybernews content on YouTube.
Your email address will not be published. Required fields are markedmarked