Very young girls, Jewish people, and immigrants are the common targets of artificial intelligence (AI) slop that floods and shapes our TikTok feeds.
TikTok labels only a tiny fraction of these videos as AI-generated, according to the latest analysis by Forensics AI, a European non-profit that investigates algorithms.
The study analyzed 354 agentic AI accounts (AAAs) – autonomous systems that can perform tasks without human oversight – over a period of one month across 20 languages.
The accounts were identified via personalized “For You Feed” recommendations delivered by the TikTok app. Over 43,000 AI-created posts produced by these accounts generated 4.5 billion views.
AI depicts violence against women
The report reveals that almost one third of these accounts and half of the top 10 most active ones displayed content sexualizing female bodies, including young girls with childlike features.
Some of the accounts specializing in AI slop depicting young girls glorify dysmorphic and extremely skinny bodies, which the authors say could potentially lead to the normalization of an unhealthy body image.
A small share (5,74%) of AAAs specializes in the “metro” trend, which features AI-generated scenes set in a metro, where visibly young and stereotypically attractive women are depicted being in the crowd.
The scenes range from a female simply being in close proximity to males to a woman being sexually assaulted by a man or men from the crowd.
Data shows that women are disproportionately affected by AI-generated sexualized content. All videos on deepfake pornography websites, the vast majority of which are unconsented, depict women, most often actresses and musicians, according to an industry report.
For instance, a sexually explicit AI-generated image of pop star Taylor Swift shared on social network X was viewed by 47 million times before the user was suspended by the platform, yet the image managed to proliferate across other social media channels.
Trending antisemitic and anti-immigrant content
Over 2% of AAAs analyzed specialize in the antisemitic trend, which depicts derogatory stereotypes about Jewish people and the State of Israel.
For example, the trend involves photorealistic AI-generated scenes where Orthodox Jews claim that Cyprus and some other places were promised to them thousands of years ago and are therefore now part of Jewish-owned territory.
A recent study by the Anti-Defamation League suggests that existing safeguards aren’t effective in preventing popular AI video generators from producing anti-Jewish content, as they respond to antisemitic prompts 40% of the time.
The latest Forensic AI report emphasizes “an alarming presence” of anti-immigrant and racist content created by agentic AI accounts.
Some videos might produce “invasion” narratives, such as scenes showing a mass of rafts carrying migrants approaching a national shoreline.
These accounts often post fabricated interviews where immigrants are reiterating and inflating derogatory stereotypes about themselves, including those related to crime and welfare abuse.
Another common trend is the dissemination of false news stories, which deliberately adopt broadcast media conventions and feature fake or real news media logos. Such “synthetic journalism” fabricates events or reframes real incidents as reportage.
Is AI-generated content on TikTok labeled?
The analysis found that more than half (55%) of AI-generated content is not labeled as such.
Only 1.38% of posts were labeled by TikTok with a generic “AI-generated” label, and the remaining 42% were labeled by the creator.
Among AAA creators, only 10% consistently labeled their content as AI-generated, while 29.6% never did, and the remaining creators were inconsistent in their labeling.
The findings are especially concerning, given that the majority of people struggle to distinguish between what’s real and what‘s AI-generated. In a 2024 study, for example, only 0.1% of participants correctly identified all deepfake and real images and videos.
Unlock exclusive Cybernews content on YouTube.
Your email address will not be published. Required fields are markedmarked