HN Explorer

Ask HN: Does AI watermarking present a new attack vector?

4 points by nathanfig ↗ HN
One thing I have not been able to determine from Claude's documentation on watermarking[0] is what kind of metadata they store in association with a given watermark. Ostensibly they could make the fingerprints as unique as they want, possibly down to the exact time, user and session.

If so, this seems like hidden risk that AI users are probably not considering. Any code you write now carries information that you might not want revealed. If a bad actor gets the keys then they may be able to de-anonymous open source contributors who want to stay hidden. Or perhaps enough fingerprints across a sample could reveal internal organization details a company would rather not disclose.

Someone with more imagination can probably come up with better examples, it just seems like an attack vector that I haven't seen much consideration for.

[0] https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content

7 comments

[ 2.3 ms ] story [ 22.0 ms ] thread
armchairhacker
If you really care, a) use an open model or b) obscure your identity to Anthropic.

Otherwise, you're already vulnerable: Anthropic can silently record every change Claude makes to every codebase, maybe they already do.

[deleted]
(comment deleted)
jjgreen
Come now, this isn't about attack vectors, it's about you getting caught using LLMs where you shouldn't.
Wpnx330
Hmmmm. If every model vendor starts training their models to watermark their content. It could become a real issue for privacy.

local cleaned models would become even more important. No longer just to cut the token bill, or allow enterprising individuals to host adult content sites as a business.