We think we've seen tech revolutions before. This expert says we're dead wrong

Are we right to believe that the AI age will usher in progress similar to the printing press, the Industrial Revolution, and the internet? An Oxford scientist thinks not, because of its sheer speed, scale, persistence, economic impact, and accountability.

We shouldn’t be too comfortable with these analogies, as they give us a false sense of security. It can feel as if we already know how to deal with yet another big technology shift coming our way.

I met Dr. Brianna Rosen, executive director of the Oxford Programme for Cyber Technology Policy at the University of Oxford, and research director for Frontier Security at the Institute for AI Policy and Strategy, ahead of a recent AI Summit in Dublin.

Rosen delivered a half-hour speech at an informal dinner with journalists from across Europe. It wasn’t exactly a crack of doom type monologue, and still, it left me with an uneasy feeling that “solving AI” might be harder than we thought.

Rosen, who previously served at the White House National Security Council and Office of the Vice President, said that one of the more difficult questions to solve with AI is alignment. How can we make sure that AI agents are aligned with human intent?

If you talk to the top experts in AI, they will give you 5 different answers about whether you can solve the alignment, when you can solve it, and if it's even possible to solve. So we genuinely don't know whether we can solve this problem. And this is like a big question in terms of how we will be able to deploy agents safely and securely going forward,

Brianna Rosen said.

Autonomous hacking incidents carried out by frontier AI models are clear signs that AI systems are not fully under human control (or don’t “understand” human intent).

AI deployment becomes extremely sensitive in intelligence and military settings. AI errors in intelligence analysis or military decision-making could escalate into a major conflict.

“You might have seen the recent news about how the US almost had a military confrontation with China over a ship that was alleged to be carrying nuclear material. And they were actually due to put planes in the air, and they were ready to board the ship. But they realized that the intelligence report on which this was based was completely an AI illusion,” Rosen said.

How do we make sure AI doesn’t start hallucinating, destroying, or inventing facts?

Here are a few other points that Rosen made:

• Companies cannot solve the problem alone: While commercial firms should do more on security, they cannot address the full challenge without government support.

Companies can't do this alone without government assistance, without government intervention, without some kind of incentive structure in place to make this happen.

• Public pressure and civil society matter: Rosen called for journalism, civil society, and the public to push leaders toward stronger action.

I think that there is a huge role for journalism, for civil society, for the public to step in and to start questioning leaders to take more action in the space.

• Governments lack visibility and capacity. Rosen criticized the limited government visibility into frontier AI systems, including models that are not publicly released. She also noted bureaucratic inertia and a shortage of technical talent in government.

They just don't have the capacity for oversight, even when oversight is so desperately needed. So, where does this leave us? I think it leaves us in a pretty scary place.

This past week, the PHP flaw might have stolen much of the spotlight, but here are a few more curious cases that Cybernews published on what’s happening around AI.

1. Professor Geoffrey Hinton, also popularly called the “Godfather of AI,” warned that humans need to “really care” about AI chatbots and their well-being, or civilization could be “toast” in the near future.
2. Major American banks are blocking third-party agents from using their services for fear that customers will suffer grave financial losses.
3. Google came under fire in Finland after around 250,000 trees were cleared from 330 hectares of forest near the small town of Muhos.
4. The US Army is reportedly developing AI agents that could infiltrate enemy networks, hunt down targets, and disable them with little human oversight. The project marks another step in Washington's push to expand its offensive cyber operations.