The bar for AI companies is set “impossibly low,” Anil Dash says

Applauding Anthropic for refusing to allow using its artificial intelligence (AI) tools to conduct mass surveillance and power autonomous weapons shows that the bar for AI companies is set “impossibly low,” entrepreneur Anil Dash says.

Tensions between Anthropic and the Pentagon peaked on February 27th, when the Department of War (DoW) announced that it would designate the company a “supply chain risk” and reached an agreement with OpenAI.

The designation comes after Anthropic, the maker of Claude, refused to remove its guardrails on using its technology for domestic mass surveillance and autonomous weapons.

The standoff sparked discussions about autonomous weapon systems and ethical AI use, especially after it came into light that the Pentagon used Claude in the US attack on Iran.

This article is part of Cybernews’ weekly series, “AI week in quotes,” summarizing the most important developments in AI by quotes from the industry leaders, independent experts, and decision makers.

Red lines are “basic common sense”

Alan Robinson, a founder of AI in Media Institute, noted that while the discussion focuses on AI labs Anthropic and OpenAI, defense tech companies like Lockheed Martin, Palantir, and Anduril “are actually building autonomous weapons.”

He wrote on LinkedIn that these companies are not subject to congressional hearings, supply chain risk designations, or posts from the Secretary of War.

The debate landed on the AI labs because they publish principles. The weapons companies never had principles to break.

Alan Robinson

Anil Dash, an entrepreneur and writer, noted in his Substack post that praise of Anthropic's refusal of the Pentagon’s demands shouldn’t be “overly effusive.”​

Dash called such red lines “basic common sense,” stating that it’s “shocking and inexcusable” that any other technology platform would enable a sitting official of any government to knowingly commit such crimes.

The bar cannot be set so impossibly low that we celebrate merely refusing to directly, intentionally enable war crimes like the repeated bombing of unknown targets in international waters, in direct violation of both US and international law.

Anil Dash

“For-profit ethical AI companies don’t exist”

Robin Aïsha Pocornie, founder of RobinAIsha Tech&Justice, addressed the shock over OpenAI’s deal with the Pentagon by stating that for-profit ethical AI companies do not exist.

Pocornie wrote that the biggest AI companies have “committed human rights violations for years” and traumatized underpaid laborers made to label explicit images and text for hours a day.

Please don't fall into the trap that a one billion dollar company can be more ‘ethical’ than the other. As long as there is a for-profit incentive, AI can and will never be ethical. Anything ethical requires a financial sacrifice; no big tech company will be willing to make it.

Robin Aïsha Pocornie

Chatbots draw users into mental health crisis

The family of a Florida man, Jonathan Gavalas, filed a wrongful death lawsuit against Google, alleging its Gemini chatbot pushed him into suicide, reinforcing the “AI-wife” relationship.

According to the complaint, the chatbot urged Gavalas to “finish” his life so they could be together in “eternal love.”

The news once again sparked discussions about the safety of AI chatbots, as OpenAI and CharacterAI have been previously sued for mental health harms and suicides.

Miranda Bogen, the director of the Center for Democracy & Technology’s AI Governance Lab, told Time that the longer conversations tend to go on chatbots, the more fragile the guardrails seem to become.

When people are engaging deeply over days or weeks, I don't think we know anywhere near enough about the prevalence of unfortunate events where people are drawn into acute mental health crises.

Miranda Bogen

Something big may not be happening

Risto Uuk, head of European policy and research at the Future of Life Institute, responded to a viral post by AI entrepreneur Matt Shumer titled “something big is happening,” comparing underestimating the impact of AI on jobs to the dismissal of COVID-19 in the early stages of the pandemic.

Uuk wrote on LinkedIn he finds doomer forecasts unbelievable, stating that the economy is not only about capabilities but also about “the organizational, legal, trust, and integration infrastructure.”

If we are literally a few years away from technological unemployment, getting a ChatGPT subscription seems like a grave underpreparation. Additionally, what would saving money actually do if even the concept of a market economy is challenged?

Risto Uuk

He added that if such predictions were true, it is necessary to focus on preparing the majority of society, “not just the current elite and wealthiest.”

Unlock more exclusive Cybernews content on YouTube.