Top hacker is a bot, yet humans still steer cyber defence

XBOW is the top hacker on HackerOne’s leaderboards, but it’s not human. Instead, it's a so-called hackbot, leading some to believe that AI-driven hackers now have the edge over real people. Fortunately, at least for now, these AI systems are limited to targeting less complex vulnerabilities. But they're doing it at scale.

XBOW has already discovered 255 vulnerabilities. However, it‘s not a sentient machine that autonomously hunts for flaws and claims bug bounties – we’re not there just yet. In reality, XBOW is an offensive security company that deploys specialized AI agents to find vulnerabilities.

Thanks to automated applications, defenders can now discover and fix flaws at scale. According to HackerOne, valid vulnerabilities on the platform have increased by 12% over the past year. In total, over 78,000 valid issues were found, with 27% of them classified as high-severity or critical. However, it’s not the bots discovering flaws that matter – it’s the people.

I virtually sat down with Michiel Prins, co-founder of HackerOne, to discuss the future of hackbots and human hackers.

“Hackbots don't sleep – they can think and react very fast. They tend to find things rather quickly, which means it's easy to lead in terms of volume. But when we look across the platform at hackbots versus more human-first hacking, you still see the humans lead in severity. They find the bugs that really have a critical business impact,” he told me.

However, this is not set in stone – humans who discover impactful vulnerabilities often use some form of automation as well.

“They’re definitely using AI in parts of their hacking workflow just to speed things up. Maybe when they wake up, they have a bunch of sort of leads and signals to follow and track down,” Prins said.

He has a term for them: "bionic hackers." According to him, they have a very human brain, but still use AI tools.

“Hackers are more productive. They're finding more vulnerabilities, which is great. They earn more on the platform. We help our customers eliminate more vulnerabilities, which is a win-win situation for everybody,” Prins said.

While individual payouts for severe and critical vulnerabilities can reach tens of thousands of dollars, the average bug bounty is $1,116. According to HackerOne, that’s a 10% decrease from 2021, when the average bounty was $1,246.

Of course, the increasing reliance on AI tools is not without challenges. For example, hackers can use AI to submit convincing vulnerability reports, some of which are exaggerated or entirely invalid.

Some vulnerabilities can be hallucinations, too.

“Six months ago, it was way worse than it is today. Actual hallucination is getting less and less common. But what is getting more common is overstating things. Because these models tend to be very pleasing. If you ask it a question, it's going to give you the most satisfactory answer,” Prins said.

This is quite the opposite of what you want from a vulnerability report.

“In a vulnerability report, you want to remove all emotion,” Prins continued.

“You want to be as factual as possible. Here's how it works. This is what I found. Here are the steps to reproduce it so you can replicate it. And here's the business impact without making up anything that makes it sound grander than it really is.”

That sounds somewhat hopeful – that humans are still very much needed in the loop.

“When you pick a target and you point the hackbot at something, there needs to be a human there to do that. Ideally, in most cases, that human provides as much context about the application as possible. The more context an AI agent has, the better it's going to perform,” Prins said.

Any finding by a hackbot needs to be validated by a human to ensure that companies can act on legitimate security issues.

“We don't want you to send in a hallucinated vulnerability that, when we try to reproduce it, there's nothing there. We do want you to take accountability for that. As an operator of a hackbot, you should only send us the things that truly matter,” he explained.

But for how long are we in the lead? Technology is advancing really fast, after all.

“Some of the most interesting high-impact vulnerabilities come from broken business logic. You can manipulate something that you weren't able to manipulate, and it changes something in a business process that has a devastating impact downstream somewhere. Those are very hard vulnerabilities for an AI to find,” Prins said.

If humans are to keep up with the bots in the future, we obviously need to understand that business logic ourselves.