AI browsers can bypass media paywalls, study finds

Artificial intelligence (AI) browsers, such as OpenAI’s Atlas and Perplexity’s Comet, can bypass media paywalls, further escalating tensions between big tech and publishers.

With the introduction of Atlas in late October, OpenAI joined the growing list of companies offering AI-powered or agentic browsers, which differ from typical browsers due to their ability to perform tasks such as responding to emails and shopping autonomously.

Bypassing media paywalls may be another distinctive feature, according to a recent analysis by the Columbia Journalism Review (CJR).

The study found that Atlas and Perplexity’s Comet browsers were able to retrieve the full text of a subscriber-exclusive article in the MIT Technology Review.

When researchers issued the same prompt to OpenAI ChatGPT’s and Perplexity’s standard interfaces, both responded that they could not access the article because the MIT Technology Review had blocked the companies’ crawlers, a type of bot.

According to CJR, AI browsers were able to read paywalled content because, for a website, an AI agent is indistinguishable from a person using a standard Chrome browser.

Automated systems like crawlers and scrapers identify themselves using a digital ID when visiting a website, and many publishers block certain crawlers to prevent generative AI tools from using their content without permission.

However, AI browsers like Comet and Atlas appear in site logs as regular Chrome sessions. Therefore, blocking them might also prevent legitimate human users from accessing a site.

Many publishers, including MIT Technology Review, use a client-side overlay paywall, meaning that the text loads on the page but is hidden behind a pop-up that prompts the user to subscribe or log in. While the content is invisible to humans, AI browsers can read it.

The CJR analysis suggests that Atlas avoids reading content from media companies that are currently suing OpenAI, a trend not observed with Comet.

Nevertheless, Atlas used various workarounds to satisfy the prompts, for example, producing a composite summary based on tweets about the article, citations in other outlets, and related coverage across the web.

Publishers see drop in traffic

The findings come as publishers are increasingly frustrated with AI using their copyrighted content without permission and compensation.

AI search engines send 96% less referral traffic to news sites than traditional Google search, Forbes reports. As traffic directly affects revenue through advertising, its reduction puts additional pressure on news media companies, which already have tight budgets.

“Zero-click” searches, where users get their answer on the results page without clicking any link, are on the rise and accounted for about 69% of Google searches by May 2025, according to a recent ImaginePro report.

Multiple media companies, including The New York Times, PCMag, and Mashable, have sued OpenAI for copyright infringement, alleging that the company used their articles to train large language models without permission.

Perplexity has recently launched a revenue-sharing scheme, which enables Comet’s users to access paywalled content from news organizations, including CNN, Fortune, and The Washington Post.

Unlock more exclusive Cybernews content on YouTube