Community Notes work on social media under one condition

A new study suggests the content moderation tool adopted by Elon Musk and Meta can work, but not by itself.

Slashing fact checker numbers and increasing the visibility of ordinary users on content, in the form of Community Notes submitted by rank-and-file users, is becoming the new norm for how tech companies seek to tackle the challenge of questionable content on their platforms.

But a new study from the University of Copenhagen suggests that one can’t really work without the other – an issue given that Community Notes are being implemented instead of traditional fact-checking processes.

The researchers analyzed more than 1.5 million community notes from X, formerly Twitter, to try to understand how effectively community moderation alone addresses misinformation. The goal was to find out whether the planned new direction big platforms are taking is the most fruitful one to tackle mis- and disinformation at their source, and to stem the flow of fake news around the web.

The team used LLMs to categorize the topics and sources cited by Community Notes, finding that professional fact-checkers play a much more significant role in successful community moderation than previously understood.

A reliance on reality

“Community Notes cite fact-checking sources up to five times more than previously reported,” the authors write.

Something like one in 20 English-language Community Notes explicitly references professional fact-checking sources when debunking or correcting a post. The proportion of notes increases by nearly 50% (to 7% of all notes) among notes rated as “helpful” by the community, compared with just 1% among notes labeled as "not helpful."

Reliance on fact-checkers becomes even more vital when looking at conspiracy-linked posts. Such claims were twice as likely to reference professional fact-checking compared to posts addressing basic misinformation, or posts with easily disproven factual errors.

“Fact-checking is especially crucial for notes on posts linked to broader narratives, which are twice as likely to reference fact-checking sources compared to other sources,” according to the researchers.

Countering the narrative

The findings are a hammer blow to tech companies that are effectively trying to crowdsource their content moderation and fact-checking services. Meta has announced it was about to phase out partnerships with fact-checking organizations in favor of relying exclusively on community moderation.

The decision by the company behind Facebook, Instagram, WhatsApp, and Threads followed increased political pressure and accusations of bias and censorship – which could be seen as undermining the mechanisms that support Community Notes’ effectiveness.

The academics behind this study suggest limiting the ability of professional fact-checkers to do their jobs by cutting funding – many are financially supported by big tech firms – could significantly hamper community moderation efforts.

“The move by platforms to end their partnerships and funding for fact-checking organizations will hinder their ability to fact-check and pursue investigative journalism, which community note writers rely on.”

“The move by platforms to end their partnerships and funding for fact-checking organizations will hinder their ability to fact-check and pursue investigative journalism, which community note writers rely on,” the authors write.

“This, in turn, will limit the efficacy of community notes, especially for high-stakes claims tied to broader narratives or conspiracies.”

The study finds that professional fact-checkers are the backbone for those amateur fact-finding efforts, providing detailed, credible analyses that users rely upon, especially when countering complex misinformation about critical issues like health, politics and scientific claims.

As the researchers themselves say: “Community notes and professional fact-checking are deeply interconnected – fact-checkers conduct in-depth research beyond the reach of amateur platform users, while community notes publicize their work.”

Without that first pass at whether something is true or not, the risk that misinformation and disinformation continue to spread on social media is constantly amplifying, with the knock-on effects that can have on our society’s discourse.