Anthropic’s scientist quits saying “world is in peril,” will write poetry instead

The artificial intelligence (AI) industry has been hit with multiple high-profile departures, with resigning researchers voicing concerns about ads in AI and pressure to set aside “what matters the most.”

Zoë Hitzig, a former research scientist at OpenAI, explained her resignation in an op-ed published by the New York Times, citing the company’s decision to introduce ads in ChatGPT.

While she emphasized that she didn’t believe ads are “immoral or unethical,” and called it a potentially “critical source of revenue,” Hitzig admitted having “deep reservations” about OpenAI’s strategy.

She said advertising built on ChatGPT’s “archive of human candor that has no precedent” creates a potential for manipulating users “in ways we don’t have the tools to understand, let alone prevent.”

Hitzig warned that OpenAI was repeating the mistakes of Facebook, which, in its early days, promised users would control their data but later broke that commitment.

“The erosion of OpenAI’s own principles to maximize engagement may already be underway,” she wrote.

“Setting aside what matters most”

OpenAI’s decision to launch ads in ChatGPT has raised concerns about privacy breaches and potential user manipulation. Anthropic, the owner of Claude, ridiculed the competitor’s move in its Super Bowl ads, provoking a bitter reaction from OpenAI’s CEO Sam Altman.

However, Anthropic was also shaken by a public resignation the same week, which put AI-related concerns in the spotlight.

The departee, a security researcher Mrinank Sharma, worked on understanding AI sycophancy, developing defences against AI-assisted bioterrorism, and learning how AI assistants could distort humanity.

In his letter to colleagues, also shared on X, Sharma said he had achieved what he wanted in the company, but also admitted he continuously finds himself “reckoning with our situation,” claiming that “the world is in peril.”

“And not just from Al, or bioweapons, but from a whole series of interconnected crises unfolding at this very moment. We appear to be approaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, lest we face the consequences,” he noted.

Have thoughts about this topic? Others do, too. Join them in the discussion.

Sharma spoke of the difficulty of letting values govern actions at both the individual and organizational levels, where “we constantly face pressures to set aside what matters most, and throughout broader society too.”

The researcher said he would explore a poetry degree and devote himself to “the practice of courageous speech.”

Two xAI co-founders left in a week

Two xAI co-founders, Tony Wu and Jimmy Ba, have also recently announced that they’re stepping down. They didn’t explain the reasons for their departures and thanked Elon Musk in their public posts.

However, the Financial Times reported that Ba's resignation followed internal tensions within its technical team over improving the model’s performance.

Musk said xAI was reorganized to improve execution speed, which required “parting ways with some people.”

Unlock more exclusive Cybernews content on YouTube.