# Swap a quarter of the words and OpenAI's watermark detector mostly stops seeing AI text

> OpenAI built textGrain for the EU AI Act. Its own numbers show detection falling to 17% after heavy synonym swaps and struggling on short passages. Whether that meets the law is unclear.

- URL: https://devisdead.ai/news/swap-a-quarter-of-the-words-and-openai-s-watermark-detector-mostly-stops-seeing-
- Published: 2026-11-05T19:09:43Z
- Author: DevIsDead Newsroom
- Section: Policy
- Tags: openai, watermarking, eu ai act, textgrain
- Originally reported by IEEE Spectrum (https://spectrum.ieee.org/ai-watermark-openai-textgrain-europe).

The EU now wants AI-generated text marked so it can be recognized. OpenAI's answer is textGrain, a watermark it released in October. Its own test numbers suggest the mark wears off quickly, Anna Desmarais reports for [IEEE Spectrum](https://spectrum.ieee.org/ai-watermark-openai-textgrain-europe), and nobody can yet say whether that is good enough for the law.

## How textGrain works

Text watermarks work by nudging word choice. A secret key produces pseudorandom values that tilt which token the model picks, leaving a statistical pattern a detector with the same key can find. Xiang Li, a University of Pennsylvania postdoc and co-author of the textGrain technical report, says the difference here is that it treats entropy, the unpredictability in a model's choices, as a budget. Developers set how much of it the watermark may spend, trading detectability against variety in the output.

## Where it breaks

The detector is tuned for a 1% false-positive rate. Length matters a lot: on test answers drawn from the ELI5 dataset, it caught about 80% of 200-token passages and 95% of 400-token ones. Li calls weak detection on short text a basic limitation of the technique.

Editing hurts more. In OpenAI's evaluation of 400-token passages, swapping 10% of the words for synonyms cut detection from 92% to 66%. Swapping up to a quarter cut it to 17%. Because each mark depends on the tokens before it, one changed word can disturb the signal in several places.

OpenAI itself says watermarking alone can't prove text is AI-made, and positions textGrain as one layer alongside content credentials and Google DeepMind's SynthID for images and audio. A detection hit also doesn't say how much a human contributed, and a miss doesn't prove a human wrote it.

## Does it satisfy the EU?

Article 50 of the AI Act took effect August 2 and requires machine-readable marks on synthetic text, images, audio and video, with solutions that are effective, robust and reliable "as far as this is technically feasible." The voluntary Code of Practice skips outputs under 200 tokens. There is no number for how robust is robust enough.

Thomas Souverain, an AI ethics postdoc in Paris, says the method looks sound and may beat competitors, but there's no peer-reviewed paper or public protocol yet. Outside testers at Cornell and Slovakia's KInIT have only just gotten access. If regulators decide it falls short, he says, fines can reach 7% of annual income. He wants OpenAI to open-source it so researchers can try to break it.

The practical takeaway: a watermark is a hint, not a verdict. Anyone building detection or compliance workflows should treat a negative result as meaningless and a positive one as evidence, not proof.

**[Scoreboard](/scoreboard) update:** AI +1. Light editing hides AI text from the watermark built to flag it.

Stay undead.