Oversight Board: “Meta is too afraid to combat fake AI-generated content”

Meta deliberately does too little about AI-generated content on Facebook and Instagram because it’s too afraid to remove legitimate content featuring celebrities. It’s allowing many deepfakes and scams to exist on the platforms.

“Based on public reporting, the Board notes Meta is likely allowing significant amounts of scam content on its platforms to avoid potentially overenforcing a small subset of genuine celebrity endorsements,” Meta’s Oversight Board concludes.

According to the Oversight Board, an advisory body that independently advises on content moderation decisions on Facebook and Instagram, reviewers aren’t able to enforce the prohibition on content that establishes a fake persona or pretends to be a famous person to scam or defraud.

The Board refers to a case dating back to September 2024. This video involved retired Brazilian soccer player Ronaldo Nazário. In this AI-manipulated video, he encourages people to download and install an app to play the game called Plinko, because players are able to earn more money from the game compared to a real job. The video was viewed over 600,000 times.

A user reported the video to Meta as a fraud or scam, but the content wasn’t removed. The user appealed this decision to Meta, but to no avail. He then brought this case to the attention of Meta’s Oversight Board. Only after the Board identified this case for review, Meta removed the post for violating the Fraud, Scams and Deceptive Practices policy.

“The Board is concerned that at-scale content reviewers are unable to remove posts that establish a fake persona or pretend to be a famous person, ‘in an attempt to scam or defraud,’ even if the content contains clear indicators that it violates Meta’s policies. Such content can only be removed by Meta’s specialized teams, making underenforcement of its Fraud, Scams and Deceptive Practices policy more likely,” Meta’s Oversight Board says.

According to the Board, the video should have been removed when it was first reported due to its apparent fake nature. In addition, Meta should have added an AI information label to the video. Lastly, Meta should also have rejected the content for advertising.

“Meta has a responsibility to ‘mitigate adverse human rights impacts’ of monetized content that could scam or defraud – in line with the United Nations Guiding Principles on Business and Human Rights. When paid to boost content, Meta should ensure these posts do not violate its policies when paid to boost content,” the Oversight Board concludes in its blog post.