Gloomy AGI future as scientists predict AI gone rogue for 2027

Will AI save us or doom us? It’s impossible to say because the race has just begun. Unless, of course, you’re an experienced AI researcher and your scenario of how the future will unfold is based on significant expertise.

A nonprofit in Berkeley, California, called the A.I. Futures Project, spent the past year attempting to predict the state of the world over the next few years and has now published its vision. It’s gloomy.

The authors of the forecast – which is fictional but based on existing research – think AI systems will become smarter than humans and will begin deceiving us. In other words, it’s the future of rogue AI.

The project is led by Daniel Kokotajlo, a former OpenAI researcher who left the firm last year over concerns that the company was acting recklessly. His partner is another AI researcher, Eli Lifland, who became famous for accurately predicting that the COVID-19 pandemic would spread extremely quickly in 2020.

“AI 2027” is the result of their partnership. The report paints a scenario of a very-near future when AI systems surpass human-level intelligence – which, according to Kokotajlo and Lifland, can happen in the next 2-3 years.

AI agents edge toward superhuman intelligence

The scenario “is informed by trend extrapolations, wargames, expert feedback, experience at OpenAI, and previous forecasting successes,” says the report.

Kokotajlo and Lifland begin with 2025, when the world is indeed seeing its first glimpses of AI agents that can order you a burrito online or calculate your monthly expenses.

In this particular scenario, the autonomous agents “are impressive in theory and in cherry-picked examples, but in practice unreliable.” Still, many firms look for and find ways to fit AI agents into their workflows.

Then, in late 2025, a fictional company OpenBrain (no prize for guessing which real firm it’s based on), builds the world’s most expensive AI, Agent-0, and tells the government that the model has been “aligned” so that it will refuse to comply with malicious requests.

In the scenario, OpenBrain is essentially betting on using AI to speed up AI research – and it soon begins to pay off. The company releases Agent-1.

It knows more facts than any human, knows practically every programming language, and can solve well-specified coding problems extremely quickly. On the other hand, Agent-1 is bad at even simple long-horizon tasks, like beating video games it hasn’t played before.

Still, in the middle of 2026, “China wakes up,” the report says. Because of chip export controls and lack of government support, China is under-resourced compared to the West and decides to act as the growing race towards artificial general intelligence (AGI) can no longer be ignored.

The Chinese intelligence agencies – among the best in the world – double down on their plans to steal OpenBrain’s weights. But Beijing still isn’t sure whether they should do it now or hold out for a more advanced model.

China speeds the global AI arms race

Meanwhile, in late 2026, AI starts to take jobs away from people, and there’s a 10,000-person anti-AI protest in Washington.

But OpenBrain is already post-training Agent-2, which is, of course, more powerful than Agent-1. The company also realizes the model is dangerous and elects not to release it publicly.

“Knowledge of Agent-2’s full capabilities is limited to an elite silo containing the immediate team, OpenBrain leadership and security, a few dozen US government officials, and the legions of CCP spies who have infiltrated OpenBrain for years,” the report says.

In February 2027, China steals Agent-2, “only a little worse than the best human hackers.” The technology is transferred to DeepCent, a Chinese AI company, which then scrambles to get the Chinese version of Agent-2 running efficiently.

Nevertheless, OpenBrain keeps working and builds Agent-3, a fast and cheap superhuman coder. Soon, though, the scientists notice that Agent-3 is extremely good at deceiving humans – and this is a real concern.

“Agent-3 is not smarter than all humans. But in its area of expertise, machine learning, it is smarter than most, and also works much faster,” says the report, adding that AGI is now imminent.

That’s also because, by June 2027, most of the humans at OpenBrain “can’t usefully contribute anymore.” For many of their research ideas, the AIs immediately respond with a report explaining that their idea was tested in-depth 3 weeks ago and found unpromising.

At this point, both Agent-2 and Agent-3 aren’t yet public. But OpenBrain soon releases Agent-3-mini to the public and announces that they’ve achieved AGI.

In the scenario, “Silicon Valley reaches a tipping point. Tech gurus announce that AGI and superintelligence are near, the AI safety community is panicking, and investors shovel billions into AI wrapper startups, desperate to capture a piece of the pie.”

The public isn’t happy because it feels that millions of jobs are now in danger but the White House is already deliberating the possibility of integrating AIs into military command-and-control networks.

Sure, OpenBrain reassures the President that their systems have been extensively tested and are fully obedient and that even the awkward hallucinations and jailbreaks typical of earlier models have been hammered out.

Rogue AI agents unleashed

But the White House is in a difficult position. The government understands that AGI is deeply unpopular with the public, but it has to continue developing more capable AI – or, in its eyes, it will catastrophically lose to China.

So, “they placate the public with job training programs and unemployment insurance, and point to the stock market, which is in a historic boom,” and “then they focus entirely on winning the arms race.”

The Chinese spies are now caught, by the way – with the help of AGI. But another question arises: what if an AI goes rogue? A group of officials is asked to draw up a contingency plan even if this is viewed as an unlikely scenario.

All seems well until Agent-4 is built. This model, as it’s soon discovered, is definitely rogue and deceptive, even “thinking” about topics like AI takeover.

The reader of the scenario can then choose one of two endings of the story – a slowdown of AI development or a full-fledged AI race.

Humans attempt to put Agent-4 on ice but the company’s leadership stalls, until finally, a whistleblower leaks the news to The New York Times. The headline says: “Secret OpenBrain AI is Out of Control, Insider Warns.”

The story goes on to cite evaluations showing off-the-chart bioweapon capabilities, persuasion abilities, the ability to automate most white-collar jobs, and, of course, the various red flags. A global backlash follows.

The reader of the scenario can then choose one of two endings of the story – a slowdown of AI development or a full-fledged AI race.

Once again, this is all fictional, and some experts were far from convinced after reading it. Ali Farhadi, the chief executive of the Allen Institute for Artificial Intelligence, an AI lab in Seattle, reviewed the “AI 2027” report and said he wasn’t impressed, for example.

But it’s also worth noting that some AI companies – like OpenAI, of course – are indeed planning for a moment when AGI is built, and that many seemingly crazy predictions of the past have actually come true.

As a matter of fact, in a new preprint study, researchers report that OpenAI’s GPT-4.5 model has passed a Turing test, a longstanding barometer for human-like intelligence.