Reddit, Yahoo, Medium are adopting new protocol to prevent free content scraping

Famous content-publishing websites, including Reddit, Yahoo, wikiHow, and Medium, among others, announced their support for the launch of the RSL Standard licensing protocol that allows publishers to set compensation, licensing, and usage terms for AI crawlers and agents.

Really Simple Licensing is a decentralized protocol based on RSS (Really Simple Syndication) standard. According to the press release, it can scale to millions of websites, applying to various forms of content, including blogs, books, videos, and datasets.

It enables publishers to add machine-readable licensing and royalty terms to their robots.txt files, which lay out specific compensation terms for AI applications and agents wishing to use their content.

“RSS was critical to the Internet’s evolution as an information ecosystem, giving early online publishers a simple, open standard to syndicate their content and reach audiences at Internet scale. That spirit of openness is what helped the web thrive,” said Tim O’Reilly, CEO of O’Reilly Media.

“But today, as AI systems absorb and repurpose that same content without permission or compensation, the rules need to evolve. RSL builds directly on the legacy of RSS, providing the missing licensing layer for the AI-first Internet. It ensures that the creators and publishers who fuel AI innovation are not just part of the conversation but fairly compensated for the value they create.”

RSL offes various licensing models, including free, attribution, subscription, pay-per-crawl (publishers get compensated every time an AI application crawls their content), and pay-per-inference (publishers get compensated every time an AI application uses their content to generate a response).

The terms can be defined and set by each publisher individually. So far, Reddit, People Inc., Yahoo, Internet Brands, Ziff Davis, Fastly, Quora, O’Reilly Media, and Medium are among the first supporters of the protocol, which is already available to any website for free.

The development of the RSL Standard is led by the RSL Technical Steering Committee (TSC), with representatives including Eckart Walther and Ramanathan V. Guha, who co-created the RSS standard.