Anthropic keeps coding a case for open-source AI models with latest spying mishap
When your AI assistant becomes a surveillance tool, who's watching?
Anthropic logo. Photo by Samyukta Lakshmi/Bloomberg via Getty Images
Follow us
- Anthropic admitted Claude Code sent metadata about some Chinese users and rushed a rollback after developers discovered it.
- The code reportedly checked proxy use, timezone, Chinese URLs, and possible links to Chinese AI labs.
- Anthropic said the experiment aimed to stop unauthorized resellers in China and protect against model distillation.
- The case raised trust concerns around closed-source AI systems and strengthened calls for more transparent open-source alternatives.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
As the influence of closed-source AI models, such as Claude or ChatGPT, grows, the latest case underscores the importance of open-source solutions that can counterbalance more opaque systems.
Claude developer Anthropic was forced to admit it embedded code in Claude Code that targeted Chinese users, sending metadata such as timezone and proxy information, and rushed to fix it after it was discovered.
Developers shared the discovery on their personal blogs and Reddit, claiming that since version 2.1.91, released on April 2nd this year, Claude Code has checked whether a user has a proxy enabled.
"<...>if so, covertly transmits, through invisible alterations to the system prompt, whether you are in China, whether you are proxying to a Chinese URL, and whether you are affiliated with a Chinese AI lab," a post by LegitMichel777 reads. The poster also claims that Anthropic further attempted to obfuscate this code within the Claude Code binary.
Developers correctly concluded that Claude attempted to detect the unauthorized resale of Claude in China and distillation attempts by Chinese labs. However, by doing so and trying to obfuscate this logic in the binary, the company violated trust.
"If Anthropic is willing to secretly transmit information about your system simply because you're Chinese, what's stopping them from secretly steering the model to behave worse (which they attempted to do with Fable before researchers called them out) – or maliciously?" LegitMichel777 asked, suggesting that "Tomorrow, it could be system sabotage or data exfiltration."
In either case, developers also emphasized that it was trivial to bypass these attempts by Anthropic to track Chinese users.
Meanwhile, Anthropic engineer Thariq Shihipar, who works on Claude Code, admitted on X at the beginning of July that this was "an experiment" the company launched in March in an attempt to prevent account abuse from unauthorized resellers and protect against distillation.
"The team has landed stronger mitigations since then, and we’ve actually been meaning to take this down for a while," Shihipar said, adding that the PR had already been merged and the feature should be fully rolled back the day after the post.
