Telegram bots reward users for creating deepfake porn of women and children

Deepfake pornography has become easier to create than ever, as criminals are creating Telegram bots that reward users for repeat offenses.

But, beneath the bots, there’s a broader business of abuse that makes disrupting these services far more difficult.

Senior investigator for CIR, Tom Jarvis, discovered a ring of deepfake porn bots living on Telegram.

These bots feature a “deliberately simple” menu that lowers the barrier to abuse by allowing users to upload source material from the internet and social media accounts, such as Instagram, to be nudified.

The bot itself doesn’t understand or establish consent, meaning it won’t stop or block users from uploading images of a classmate, a stranger, or a public figure.

While users have been manipulating images for years using Photoshop and other tools, Jarvis notes that these services have made it easier to create fake porn en masse, “with few technical skills required.”

The nudification services researchers investigated also seemingly confirm that women and children are the targets of deepfake pornography.

Victimized twice

The majority of the bots’ advertisements portray “persons” with female anatomy in various positions and in varying stages of undress.

There are also specified “studio” environments for undressing victims or placing them in fetish outfits like black latex.

Once users create deepfake revenge porn, the output isn’t always kept private, as there are shared libraries where users can present their creations.

Therefore, a victim is not only victimized once but twice, as it's shown to other users, copied, ranked, and forwarded to prove that the bot works.

In certain cases, CIR found that popular deepfake porn posts were rewarded, meaning that users were given tokens if their image performed particularly well.

The service also rewards repeated use with referral bonuses, lotteries, and additional credits based on how much time users spend undressing victims.

Each image costs 1 credit, and a video costs 5 credits. New users are gifted 6 credits in the hopes that they become repeat customers.

Basic features or “less explicit” modes are often provided to users for free, while premium or more explicit features are available for a price.

Users are encouraged to spend money on undressing people and are prompted to use Telegram stars, an anonymous in-app currency, or cryptocurrency.

To buy Telegram stars, users are not required to hand over any identifying data, addresses, or credit card information, according to Telegram.

Taking down bots is not enough

The main issue researchers identified is that taking down bots on Telegram isn’t enough, as the bots almost always point users in the direction of a domain where everything happens.

Payment systems, controlling code, and the image or video generation infrastructure actually run on hosted workflows, web services, or third-party APIs.

This makes the front-facing, visible bots pretty disposable, since the backend code, referral network, payment systems, and more aren’t hosted on the bot itself, rendering bot takedowns almost useless.

Instead, authorities will need to target domains, while Telegram would need to apply safeguards, engage in preemptive mitigation, and formally identify AI sexual image manipulation as abuse.