Massive analysis of 311 million malware warnings: here’s how hackers fool us

311 million records containing 77 million URLs were analysed to develop the document.

Malware is a massive issue for internet users across the world, and one of the main ways that we’re hoodwinked is by clicking on something we’re not meant to. Of course, the way that happens is by directing us to URLs. “URLs are central to a myriad of cyber-security threats, from phishing to the distribution of malware,” say the authors of a new paper that tries to uncover and characterise what makes a maliciously-used URL.

Researchers at Australia’s Commonwealth Science and Industrial Research Organisation (CSI) researched and analysed 311 million records containing 77 million URLs that were submitted to Hispasec Sistemas’s online antivirus checking website, VirusTotal between December 2019 and January 2020. The findings were astounding for the scale of the malware issue discovered.

From the dataset the researchers analysed, a staggering 2.6 million suspicious campaigns were identified based on their attached metadata, 77,810 of which were confirmed to be malicious through a secondary check. In total, 38.1 million records and 9.9 million URLs were found within those 2.6 million malicious campaigns.

Digging into the detail

Perhaps most concerning for those trying to spot these malicious campaigns is the volume of worrying ones that slipped through the net. “Some surprising findings were observed, such as detection rates falling to just 13.27% for campaigns that employ more than 100 unique URLs,” the researchers say.

A quarter of all submissions came from the United States, with 17 million unique pieces of content analysed.

Submitted URLs were checked by a median of 72 security vendors to determine if they’re benign or malicious. But what was worrying was that almost all – 98% - of all the submissions were only flagged as malicious by 10 or fewer vendors at a time. “This indicates that vendor detection performance is highly skewed and only a few of them are effective,” the researchers write.

That may seem a major concern – and it is, given the ineffectiveness of much of the market – but the wisdom of crowds does help spot those malicious sources. The researchers’ findings are that if a URL is flagged by at least four vendors, it is reasonable to conclude that it is malicious. That means that while any individual security vendor solution may not be that reliable, the market as a whole is able to provide a safety net through strength in numbers.

A barrage of malware

Still, malware detection is a cat and mouse game, and the hackers and cybercriminals behind these campaigns know how to try and force their way through defences. And they literally do: the vast majority of malicious URLs come from campaigns that employ multiple unique URLs, according to the researchers. The goal too is to bamboozle users into thinking they’re visiting a legitimate URL when in fact it’s a fraudulent one.

The average URL lengths across campaigns (i.e., mean of means) stands at 64.29 characters, say those who parsed the data.

Those campaigns often try to mimic big brands to capitalise on the trust those brands have built up over time. Take for instance one campaign launched by hackers. One campaign of 4,081 unique URLs tried to pass off as an Apple brand. It used a combination of 9 sub-domains, 12 domains, and 7 suffixes, ranging from www.apple.com as a subdomain, icloud-com as a domain, and .us, .live, and .support as a gTLD.

“We see the efforts [cybercriminals] go to in order to evade defences, with our findings on their use of widely variant URL lengths and propensity for longer URLs,” say the researchers. “It is hoped that such insights would be of use to the wider cyber-security community.”