Security researchers have demonstrated a new type of attack that uses hidden audio signals to manipulate voice assistants into carrying out unauthorized actions without users noticing.

In one theoretical scenario, an employee joins a Zoom call where soft background music plays beneath a presentation.

To everyone on the call, the audio sounds normal, maybe just a little more of an echo than usual, but hidden inside the sound is a malicious signal that is targeting the AI meeting transcriber.

While staff continue to discuss quarterly targets, the transcriber receives covert instructions hidden in the audio itself – directing it to search for sensitive files, company secrets or to send information to an attacker-controlled email address.

Researchers from Zhejiang University, the National University of Singapore, and Nanyang Technological University call this proof-of-concept technique “AudioHijack.”

The technique was presented on Wednesday at IEEE Symposium on Security and Privacy in San Francisco.

Audio agents targeted

The attack targets the growing number of commercial AI systems from the likes of Microsoft Azure and Mistral AI that are capable of listening, speaking and interacting with external tools such as email, calendars and web browsers.

The researchers describe the method as a form of “auditory prompt injection” where malicious instructions are hidden inside audio content such as podcasts, music, videos or voice recordings.

Unlike traditional hacks, the attack doesn’t need malware or require direct access to a device, instead it hijacks the AI model itself through sound.

Room echo and natural sounds used

The technique works by subtly altering audio waveforms to make tiny, almost inaudible changes to a sound clip so that humans hear normal sound but the AI system interprets the hidden patterns as commands.

For the concept, the researchers created subtle sound modifications designed to resemble a room echo.

Have thoughts about this topic? Others do, too. Join them in the discussion.

The team tested the technique against 13 major open-source audio AI systems, including Qwen2-Audio, GLM-4-Voice, Phi-4-Mulitmodal, Voxtral-Mini and Kimi-Audio.

The researchers also demonstrated how the attacks could transfer to commercial voice agents from Microsoft Azure and Mistral AI.

They found that adversarial audio generated locally could “reliably manipulate these agents into executing authorized actions through single or cascaded tool calls.”

The success rates in the test were high – ranging from 79% to 96% across different scenarios. Among the demonstrated behaviors were:

• issuing sensitive web searches
• downloading files from attacker-controlled source
• exfiltrating user information through email

Researchers found that training the models to watch out for these suspicious prompts only reduced attack success rates by 7%, while asking systems to verify whether their responses matched user intent detected just 28% of attacks.

User overridden by hidden audio commands

While there have been previous studios on adversarial audio, lead author Meng Chen, a PhD student at Zhejiang University said that what singled out this new work was that it targets generative models capable of producing responses and taking actions.

Chen added that the attacks are especially dangerous because they do not depend on knowing what the victim is asking the AI assistant to do.

“It takes half an hour to train this signal,” Chen told the IEEE. “And because it is context-agnostic, you can use it to attack the target model whenever you want, no matter what the user says.”

Microsoft responds

Chen said the attacks could work in realistic situations involving “online videos, music clips, or voice notes that users query an AI about,” as well as malicious audio broadcast during Zoom calls that are processed by AI transcribers.

The research raises concerns as tech companies integrate voice assistants into smartphones, enterprise software and customer service platforms.

In a statement, Microsoft told IEEE that it welcomed the researchers’ work in "helping improve understanding of 'model resilience' in a controlled setting," but added that real deployments have other safeguards, and developers should use them.

“In practice, AI models are often integrated into user applications, and we offer developers tools and guidance they can use to implement additional layers of protection that help safeguard users," the company said.

---