OpenAI will watermark ChatGPT texts in EU, but there’s a catch
OpenAI’s invisible watermark is easy to bypass.
Image by Thomas Fuller/SOPA Images/LightRocket via Getty Images
Follow us
- OpenAI will add invisible watermarks to ChatGPT and Codex text for eligible EU users.
- The textGrain watermark is less reliable for short text, edited text, recipes, and math content.
- Replacing 25% of words can drop OpenAI’s watermark detection rate to 17%.
- OpenAI is limiting detector access to approved researchers and expert organizations while it tests reliability.
Key Takeaways by nexos.ai, reviewed by Cybernews staff.
OpenAI is adding an invisible watermark to texts generated with ChatGPT in the EU. But it’s not quite bulletproof yet.
OpenAI’s text watermarking technology is called textGrain, which adds an invisible statistical signal to ChatGPT’s word choices.
According to the AI company, textGrain exceeds Google’s SynthID for text and works well in many cases. However, the tool is less effective in 2 situations: when a text is short and when it has been edited.
OpenAI’s detector identified watermarks in about 80% of 200-token passages, compared with about 95% of 400-token passages. Detection rates are significantly lower for content such as mathematics or cooking recipes, where there’s less flexibility in word choice.
When only 10% of words are replaced with synonyms, OpenAI’s detection rates are reduced from 92% to 66%. When a quarter of words are edited, the detection rate drops to 17%.
That’s why textGrain currently isn’t widely available to the public.
“These limitations contribute to our decision to provide initial detector access only to approved researchers and expert organizations, who can help us evaluate reliability and responsible uses,” OpenAI says in a blog about its invisible watermarking technology.
Testing ground
Over the coming weeks, OpenAI will introduce its text watermarking technology to eligible ChatGPT and Codex users across all plans in the EU to learn from real-world experience and collect feedback.
In addition, API customers around the world will be able to opt in to watermarked text outputs for selected models. Approved researchers and expert organizations can apply for access to OpenAI’s text watermarking.
OpenAI decided to add text watermarking technology to its texts in response to the EU AI Act, which was designed to ensure AI companies act transparently.
Article 50 of the AI Act states that all AI companies must include machine-readable watermarks to show whether text has been synthetically created or manipulated.
Companies that fail to comply risk hefty fines of up to €15 million ($16.8 million) or 3% of their global annual turnover, whichever is greater.ant
