Seems pretty easy to defeat by running text output through a random reworder process that would effectively repeat the same routine on low-stakes words, replacing them with similar ones. We learned this in high school, jumping through your paper and hitting random words with the thesaurus to 'sound smarter'
> Google DeepMind tested this impact by serving a model that used watermarking to a portion of their Gemini traffic and comparing thumbs-up and thumbs-down ratings. They found no statistically significant differences from the unwatermarked model. And in a controlled study, human raters comparing watermarked and unwatermarked answers side-by-side saw no difference in quality.
For some reason I had assumed testing this would be more sophisticated than just checking the thumbs up/down stats and user "vibes"
I’d like to better understand the minimum text length to get a confident result, i would presume it would need to be quite long, perhaps > 1000 words to get an accurate result.
Interesting. Here's the section of the EU Act that mandates this:
> Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated. Providers shall ensure their technical solutions are effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art, as may be reflected in relevant technical standards. This obligation shall not apply to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof, or where authorised by law to detect, prevent, investigate or prosecute criminal offences.
> We will soon be offering a watermark detection API. We’re in the process of working out the details of its implementation.
Dumb question - doesn't this defeat the purpose of a watermark? i.e., anyone who wants to avoid detection can simply run `while (has_watermark(text)) text = slightly_rewrite_with_non_anthropic_llm(text)` until it's gone? I feel I am missing the intent of the watermark if it is so easily defeated.
The method of identifying authorship isn’t new. I guess the main new thing here is to ensure Claude has a specified word distribution so you can identify its writing.
From what I understand when you re-tokenize the output you can simply look at how often certain tokens show up and the position of them, enough of these matches would result it watermarked text.
Let's say we are at token 431 and there is 49% to generate token 1 and 51% to generate token 2, we apply bias to our token 1 which would make it win causing a repeating pattern invisible to the human eye.
Now you apply this to multiple tokens and a reversible source of random you have a pretty strong watermarking system... That is rather annoying to defeat as you essentially have to rewrite most of the text. The alternative is to use a diffusion model and spray some gaps across non-literal information such as ids, links, etc.
>But if we could see the sequence of all the moves after the game (and we knew the value of pi), we could work out whether this was a game that likely used pi to determine its moves. The game that used pi is, in a sense, “watermarked”.
Wouldn't pi contain any such sequence of numbers? Therefore you'd have to allow only certain regions of pi, and therefore, its not random anymore and we could just shortcut the whole game?
anthropic speed running its way into irrelevance. wtf would i use AI for writing that screams AI generated especially when I'm not in the EU and open models are so good now
How I use claude in my grant writing.
I write a rough paragraph. I invoke /concise-mode skill (a supposed instruction that Claude used for their previous concise writing style), and ask it to revise for clarity. I re-read to ensure it says what I wanted, ask for another revision with a specific request, or manually edit.
This is a productivity enhancement for me. I am not writing art. I am delivering information for my research plan. While I would not mind a flag that indicated AI assisted for clarity, I do not want to be accused of using AI-wholesale. I put a lot of work into it, and I do not want to be maligned.
Do i understand correctly - to check watermark you need full model weights, of all org models. So running the check is basically the same as running every model once? That's really expensive
30 comments
[ 1.1 ms ] story [ 18.4 ms ] threadIs poor proofreading a form of watermarking? Clever, I suppose, but they should consider running posts through Sol for clarity.
For some reason I had assumed testing this would be more sophisticated than just checking the thumbs up/down stats and user "vibes"
> Providers of AI systems, including general-purpose AI systems, generating synthetic audio, image, video or text content, shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated. Providers shall ensure their technical solutions are effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art, as may be reflected in relevant technical standards. This obligation shall not apply to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof, or where authorised by law to detect, prevent, investigate or prosecute criminal offences.
https://eur-lex.europa.eu/eli/reg/2024/1689/2026-07-27/eng
It definitely makes Pangram's job a bit easier.
Dumb question - doesn't this defeat the purpose of a watermark? i.e., anyone who wants to avoid detection can simply run `while (has_watermark(text)) text = slightly_rewrite_with_non_anthropic_llm(text)` until it's gone? I feel I am missing the intent of the watermark if it is so easily defeated.
https://towardsdatascience.com/text-classification-and-the-b...
Let's say we are at token 431 and there is 49% to generate token 1 and 51% to generate token 2, we apply bias to our token 1 which would make it win causing a repeating pattern invisible to the human eye.
Now you apply this to multiple tokens and a reversible source of random you have a pretty strong watermarking system... That is rather annoying to defeat as you essentially have to rewrite most of the text. The alternative is to use a diffusion model and spray some gaps across non-literal information such as ids, links, etc.
Wouldn't pi contain any such sequence of numbers? Therefore you'd have to allow only certain regions of pi, and therefore, its not random anymore and we could just shortcut the whole game?
E.g. prompt Claude to write all sentence in reverse, or swap every 2 words etc. Then use a script to put reorder in the right ordering?
This is a productivity enhancement for me. I am not writing art. I am delivering information for my research plan. While I would not mind a flag that indicated AI assisted for clarity, I do not want to be accused of using AI-wholesale. I put a lot of work into it, and I do not want to be maligned.
I believe they know damn well that this will lead nowhere, and are only doing this to mitigate criticism.
> No. The watermarking applies to Claude and its outputs. It doesn’t identify anything to do with individual users
But there's nothing stopping them from doing this in the future. In fact, couldn't they append the user's ID to the key in some manner?