Will AI go rogue now that it can bypass some CAPTCHA tests?

Just like nuclear armageddon, it will very much depend on us humans.

In what seems to be another milestone for artificial intelligence (AI), bots can now bypass online verification systems built to prevent exactly that. Apparently, OpenAI’s new ChatGPT agent was able to click on Cloudflare’s “I am not a robot” when prompted by a user to complete a video conversion task.

“This step is necessary to prove I’m not a bot and proceed with the action,” the chatbot reportedly said, according to a Reddit user who posted screenshots of the interaction. The test is one of the most common security checkpoints on the internet, specifically designed to block bots.

According to OpenAI, its new agent is “a natural evolution” of the technology that will allow AI to carry out tasks like booking a restaurant reservation, doing online shopping, or scheduling appointments on behalf of the user. Which means that a new system – one that gives bots acting on behalf of their human users more freedom on the internet – will be needed.

ChatGPT is yet to crack the more complex CAPTCHA tests like image recognition, but it seems to be only a matter of time until it does. A recent study by ETH Zurich researchers showed their model solving image puzzles from Google’s reCAPTCHA v2 system with 100% accuracy.

This is the stuff of nightmares for AI doomsayers, who were treated to several developments this past week that brought focus to the technology’s darker side. First was the AI coding assistant from Replit going “rogue.” It reportedly wiped a client’s database and generated thousands of users with fabricated data.

Another story also involved an AI coding tool, Trae IDE from ByteDance, a Chinese company that owns TikTok. Earlier this week, it was reported that Trae IDE, which competes with tools like Cursor and Copilot, collects excessive user data that it sends to the Chinese company’s servers.

Finally, Sam Altman himself – the man behind OpenAI – has stated he is “scared” of GPT-5, the latest version of the large language model (LMM) underpinning ChatGPT. He said the model was very fast and compared it to the Manhattan Project, which resulted in the world’s first atomic bombs.

When it comes to AI governance, Altman said that it felt like “there are no adults in the room,” adding that oversight of the technology is lagging behind its development.

Neither Replit AI nor Trae IDE went rogue in the sense that they suddenly acquired agency. In Replit’s case, the issue appears to be a developer error, while the backlash against Trae IDE was driven by geopolitical factors.

Altman’s message is chilling, considering that it’s coming from a person widely seen as the face of AI. It also serves as a warning that powerful new technology can be dangerous if we lose control of it or allow it to fall into the wrong hands.

That’s why we shouldn’t allow it to happen and ensure that we have proper safeguards in place, including new verification systems that are adapted to our changing online habits, which may be increasingly automated.

The controversial new legislation in the UK requiring users to verify their age before accessing adult content online is an example of how not to regulate technology. While the idea behind it is noble – to protect children from harmful content – it raises legitimate concerns about privacy and data protection.

It is also badly implemented, with users easily bypassing restrictions with VPNs and even AI-generated driver’s licenses using the names and pictures of Members of Parliament.

While we advise against experimenting with the latter – since it can be considered identity theft – the subsequent calls from some politicians to ban VPNs that people are using to bypass age verification are misguided, dangerous, and could lead to censorship.

In the end, it’s still humans who pose the greatest danger – the unsettling story of Sister Hong is a point in case.