“The game has changed:” AI-powered malware scales up attacks

Threat actors have begun utilizing generative artificial intelligence (AI) to conduct more sophisticated attacks, raising concerns about the effectiveness of standard cybersecurity practices.

A recent report by Google Threat Intelligence Group highlights a shift that occurred within the last year: government-backed threat actors and cybercriminals have been deploying AI-enabled malware in active operations.

Malware distributors have started using large language models (LLMs) like Gemini during attack execution to generate malicious scripts, obfuscate their own code to evade detection, and leverage AI models to create malicious functions on demand.

Threat actors disguise themselves as students and researchers to obtain information that LLMs otherwise would block. Generative AI is now used to enhance all stages of their operations.

Meanwhile, underground forums are increasingly offering illicit AI tools that can help low-level threat actors scale up their intrusions.

The biggest takeaway from Google’s report is that the age of “signature-based” cybersecurity protections is on the downward slope, according to Adam Arellano, a field chief information security officer and chief technology officer at Harness.

He tells Cybernews, “There will still be a market and widespread use of signature-based tools, but as more and more adversaries start to use self-changing attacks, the less helpful those tools will be.”

Cybercrime is going “machine speed at scale”

Robeson Jennings, a senior vice president at ZeroFox, says the findings confirm that generative AI continues to shift cybercrime from manual to machine speed at scale.

Bad actors are training models to probe defenses, personalize lures, and evade safeguards in seconds.

“For users, this means online deception is becoming even more ubiquitous and harder to spot – the signals and filters we once relied on to detect and mitigate a scam have become erased by automation,” Jennings tells Cybernews.

For example, the malware family, PROMPTLOCK, uses cross-platform ransomware written in the Go language, which leverages an LLM to generate and execute malicious Lua scripts at runtime dynamically, according to the report.

This may result in filesystem reconnaissance, data exfiltration, and file encryption on Windows and Linux systems. Although still in their nascent stage, such developments represent a significant step toward more autonomous and adaptive malware.

For users, this means online deception is becoming even more ubiquitous and harder to spot – the signals and filters we once relied on to detect and mitigate a scam have become erased by automation.

Robeson Jennings

John Coursen, chief information security officer and founder at Fortify Cyber, says the findings prove that the game has changed. Still, the greatest threat isn’t AI – it is social engineering amplified by AI.

“AI has become a force multiplier for the oldest hack in the book: getting people to do things they shouldn’t be doing by manipulating human trust,” Coursen says.

Looking for spelling mistakes to recognize scammers may no longer work, as attackers are using AI to generate flawless, convincing phishing lures in any language.

“The email from your ‘boss’ will be perfect. The voicemail from your family member will sound just like them,” Coursen says.

The Google report highlights that threat actors adopt social engineering-like pretexts in their prompts to bypass AI safety guardrails. For example, a China-nexus threat actor requested that Gemini identify vulnerabilities on a compromised system.

When the chatbot refused, saying that a detailed response would not be safe, the actor reframed the prompt by depicting themselves as a participant in a capture-the-flag (CTF) exercise, a gamified cybersecurity competition.

The change in prompt resulted in Gemini providing helpful information that could be misused to exploit the system. The actor continued to ask for information about the exploitation of specific software and email prompts, using participation in CTF as a pretext.

Humans, not software, are the defenders

The market for illicit AI markets sold on underground English and Russian forums has matured, according to Google’s report.

These tools and services can enable low-level actors to augment the frequency, scope, efficacy, and complexity of their intrusions, despite their otherwise limited technical acumen and financial resources.

Most of these tools are advertised for their capabilities to carry out phishing attacks, where scammers impersonate trusted individuals or companies to trick people into revealing sensitive information, such as passwords and credit card numbers.

Get a frantic voicemail from a family member asking for money? Hang up and call their phone directly.

John Coursen

Coursen says there’s no need for buying “more magic AI software or AI security appliances” to prevent phishing attacks. Instead, humans should become the defenders.

Over six in ten adults in the United States report receiving scam phone calls, emails, or text messages attempting to obtain their personal information at least weekly, according to a Pew Research Center survey.

Phishing scams resulted in an excess of $70 million in losses in 2024, according to data from the Federal Bureau of Investigation. The staggering losses underscore the sophistication of these attacks and the need for greater awareness.

Coursen says people should never verify a request using the same channel it came from. For example, a CEO’s request asking for an urgent wire transfer should be verified by making a phone call or sending a new separate message on Slack.

“Get a frantic voicemail from a family member asking for money? Hang up and call their phone directly,” he explains.

Establishing a family safe word, which should be simple, weird, and non-public, may help against phishing attacks that involve AI voice-cloning and deepfakes. If a family member cannot provide the safe word during a frantic call, you should hang up.

While continuously cleaning one’s digital footprint is often overlooked, it can help prevent users from becoming targets for cybercriminals.

Coursen tells Cybernews, “AI attackers are scraping your social media. They know your job title, your boss’s name, your colleagues, the project you just posted about, and your kid’s name.”

Unlock more exclusive Cybernews content on YouTube.