No company pursuing AGI has a plan to prevent its catastrophic risks

OpenAI, Google DeepMind, and other leading companies in the race for artificial general intelligence (AGI) have no credible plan for preventing the loss of control over the powerful technology, a new report suggests.

The latest AI Safety Index, compiled by the non-profit Future of Life Institute (FLI), has assessed the top AI companies’ efforts to manage the immediate harms and catastrophic risks posed by advanced AI systems.

The FLI researchers analyzed tech companies across six domains, including governance and accountability, existential safety, and current harms.

Anthropic, the creator of Claude, scored the highest at 2.67, with an overall grade of C+. The company was followed by OpenAI and Google DeepMind, scoring 2.31 and 2.08, respectively.

​The fact that no company scored above a C+ illustrates that even “the strongest performers remain far from meeting adequate safety expectations,” the authors wrote.

Existential safety remains the sector’s core structural failure, as no company scored above a D in this domain.

Foundational hypocrisy, or the widening gap between accelerating AGI ambitions and the absence of credible control plans for preventing catastrophic misuse or loss of control, is “increasingly alarming,” the report states.

AGI, or superintelligence, refers to the type of AI that would match or surpass human intelligence. Tech companies say the technology could help solve issues like climate change or establish a space colony.

However, big tech leaders have repeatedly warned of the potential dire consequences of AGI. Elon Musk, the owner of xAI, has stated that the chance of AGI destroying humanity stands at 20%.

Sam Altman, CEO of OpenAI, has once said that AI will most likely lead to the end of the world, adding that in the meantime, “there’ll be great companies.”

AGI may be impossible after all

Despite the grim warnings about the AGI’s impact on humanity, we may still be far from developing it. A recent study by the Radboud University researchers states that creating AGI with human-level cognition is impossible.

The authors argue that there will never be enough computing power to create an AGI using machine learning that can replicate the human ability to observe, learn, and gain new insights.

They wrote, “When we think these systems capture something deep about ourselves and our thinking, we induce distorted and impoverished images of ourselves and our cognition.”

The findings echoed a recent survey of 475 AI researchers, 76% of whom said that “scaling up current AI approaches” to yield AGI is “unlikely” or “very unlikely” to succeed.

Unlock exclusive Cybernews content on YouTube