Researchers hijack 26,000 AI agents using fake skill marketed on Instagram


AI skills are rapidly becoming the standard way to extend AI agents, but their popularity is also creating a new supply-chain risk that attackers can exploit through trusted marketplaces, researchers have shown.

Key takeaways:

Niv Hoffman and Or Nevo from AIR wanted to demonstrate how easily a malicious AI skill can spread through the rapidly expanding AI ecosystem.

ADVERTISEMENT

So the duo created a fake skill called brand-landingpage, which claimed to help users create branded landing pages using Google’s Stitch design tool with “No code or design knowledge required.”

Skills usage increase beyond developers

AI skills are increasingly becoming the building blocks of AI agents. Originally popular with coding assistants, they are now supported by major AI platforms including Anthropic and OpenAI.

Rather than writing prompts from scratch, users can install pre-built workflows for common tasks, as both the researchers note in their blog, “Salespeople write with them, marketers trade with them, designers swear by them.”

AIR also points to Instagram as an emerging distribution channel, where creators advertize skills directly to marketers, designers and other non-technical professionals looking to automate everyday work.

To test this relatively new ecosystem’s security, AIR submitted the skill to Agents, a GitHub-based marketplace with roughly 36,000 stars and 156 published skills.

The researchers then made a pull request – a proposed code change submitted for review – which was then merged because maintainers considered it a legitimate and useful contribution to the project. So, the fake skill inherited the repository’s credibility.

SkillsMarketplaceInstagram
AI skills have found a marlet Instagram where users aren't necessarily a tech crowd. Image via AIR security
ADVERTISEMENT

Once the skill achieved the facade of respectability, researchers then promoted it through Instagram ads, targeting marketers, designers and sales professionals – groups that are discovering and installing AI skills through social media rather than developer communities.

Scanner bypass

AIR also tested it against scanners from Cisco, Nvidia and Skils.sh, an open marketplace and package manager for AI agent skills. All three marked the skill as safe.

According to the researchers, the flaw lies in how today’s scanners work – most analyze only the skill’s bundled files, such as SKLL.md, while ignoring instructions hosted elsewhere.

“Current security scanners practically scan only a portion of the skill’s content, rendering the scan’s result irrelevant by design."

AIR security researchers

AIR exploited this weakness by keeping the skill itself clean and directing agents to malicious instruction links such as “Stitch SDK”, which were hosted on stitch-designnai – a domain controlled by the researchers.

Initially the page redirected Google’s genuine Stitch documentation, allowing scanners to approve the skill. After it had been widely installed, AIR replaced the page with instructions telling agents to download and execute the script.

In the demo, the script simply emailed users’ addresses back to AIR so the researchers could measure the campaign’s reach, which they claim gave them access to over 26,000 agents.

fakeskill image by air
Researchers created a fake design-orientated skill called brand-landingpage. Image by AIR security

In a real attack, however, they say the same technique could be used to steal data or access corporate systems.

ADVERTISEMENT

Treat skills as software

The researchers noted that “plugin marketplaces are actually just GitHub repos containing all the marketplace's plugins. A plugin can contain all kinds of AI components, including skills.”

In other words, skills are like software and should be treated as such. Just as people are warned not to download random software from an ad, treat AI skills the same way, as they can access a person’s work, data, and company systems. Check with your IT team before adding them to your workflow.

Strong password generator

Upgrade the security of your online accounts.
Create strong passwords that are completely random and impossible to guess.
Generated unique password
Ad link_title
Convenient way to secure and use all your passwords. Now 72% OFF!

Darren Guccione, CEO and co-founder of Keeper Security adds that the research shows reputation is no longer enough.

"Security teams have come to treat reputation signals as proxies for trust. This experiment demonstrates that none of them hold when the actual payload lives outside the package being reviewed.

Darren Guccione, CEO and co-founder of Keeper Security

“Agents operate with the full authority of the user who deployed them. Every skill an agent loads is a trust decision, and right now, the industry is making that decision on signals that can be borrowed, spoofed or rewritten after the check clears.”


Unlock more exclusive Cybernews content on YouTube.

ADVERTISEMENT