最新资讯
reddit.com OpenAI 称 Astra 仅满足 10% 安全任务标准 dev.to FreeToken:在自有 GPU 上运行前沿 MoE 模型及其内存需求 dev.to 当 AI Agent 说“完成”时,究竟完成了什么? dev.to 部署 Underdog Bark 3.0 Pro:逐字转写、自定义后端与分块设置 dev.to 为艰难对话构建 AI:五个 Agent、一条安全正则与一位盲评监督者 techcrunch.com Palo Alto Networks 洽谈以 5 亿美元收购 AI Agent 安全初创公司 Clutch theinformation.com 黄仁勋被要求就涉华间谍指控向国会作简报 www.infoq.com Edera 推出 Object Capability Bounded Authority,试图将凭证与 AI Agent 工作负载隔离 wechat 这波杀疯了!旗舰级模型不限量、1个月视频额度免费领 bloomberg.com AI 抛售蔓延至信用市场,软银领跌亚洲美元债 dev.to 如何在专用 GPU 服务器上用 OpenAI 兼容 API 部署 LLM dev.to 审计 153 份 Claude Code 与 Codex 会话记录:User-Agent 与 URL 中的邮箱泄露
多元动态
登录 注册
大模型 spectrum.ieee.org · @Matthew S. Smith

AI模型正在给文本加水印——你能察觉吗?

原文 我的收藏

Anthropic于8月11日宣布,未来所有Claude模型生成的文本将包含水印以标识AI生成内容。Google已在Gemini模型中使用类似水印,OpenAI也计划引入。此举部分源于欧盟AI法案的要求。文章探讨了文本水印的工作原理、潜在的质量影响以及专家之间的争议。

AI推荐理由 IEEE Spectrum文章详细介绍了AI文本水印技术现状,包括Anthropic、Google和OpenAI的计划,并讨论了欧盟AI法案的影响及技术争议。来源权威,信息密度高,对AI政策和技术发展有重要参考价值。

80

正文

AINews AI Models Are Watermarking Text—Will You Notice? AI text watermarks are often, but not always, undetectable Matthew S. Smith 3m 6 min read Matthew S. Smith is a contributing editor for IEEE Spectrum and the former lead reviews editor at Digital Trends. Nicole Millman On 11 August, Anthropic announced that all future Claude models will generate text that contains a watermark that identifies its results as AI generated. The company is not alone. Google has its own text watermark (which Anthropic’s is based on) it uses on the output of its Gemini models. OpenAI has yet to introduce a text watermark but it plans to do so. The rapid spread of watermarking is in part a response to the European Union’s AI Act, which mandates watermarks for AI models released after 2 August, 2026, along with other planned and proposed regulations aimed at curbing the spread of deceptive or manipulative AI-generated content. But the new rules may come at a cost for AI users who simply want the best possible results. AI watermarks can apply to many forms of content: The EU AI Act also requires them for images, audio, and video. Such media watermarks have been in use for years, and while their effectiveness as a holistic solution to marking AI remains up for debate, they can achieve detection rates above 99 percent. Image and video watermarks are already deployed by OpenAI, Google, and Meta, among others. (Anthropic doesn’t provide an image generation model.) Text watermarks have been less frequently deployed, however, and not everyone is convinced that text watermarking can work without compromising the quality of an AI model’s response. John Gruber, a prolific technology writer and co-creator of the Markdown language, calls the watermark a “perversion of writing” and disputes Anthropic’s assertion that a watermark doesn’t change the meaning or quality of text. Images consist of millions of pixels, he notes, whereas text responses often span just dozens or hundreds of words. Text seems to provide far less space to alter AI output in a way that is detectable yet not disruptive. John Kirchenbauer, postdoctoral fellow at the Vector Institute and co-author of a 2023 paper which was among the first to describe a text watermarking method, disagrees. “[A watermark] wouldn’t be detectable if there wasn’t a change. This is a very fundamental point,” he says. “The question is, do you care if it’s not the exact original distribution if, for all intents and purposes, it doesn’t change the utility to you?” Realistically, the issue comes down to that word, “utility.” Does watermarking AI-generated text meaningfully degrade the experience of the person using it? The answer is still under dispute. How AI Text Watermarks Work The term “watermark” is so familiar that it can cause confusion about how the technology works when applied to AI. A text watermark is not metadata or invisible characters; it is something much more subtle. The exact details vary between methods, but text watermarks are generally impossible for a human (and, in many cases, even a computer) to detect without access to the specific key used to detect a specific watermark. Understanding why requires an understanding of how LLMs work. An LLM produces a probability for every word that could come next at each step in its response to a prompt. (From here on, I’ll be using “words” interchangeably with “tokens,” although tokens also represent numbers, punctuation, and more). A likely word might get a 40 percent probability, a plausible alternative 10 percent, and an unlikely one a fraction of a percent. The model then picks a word at random, weighted by those numbers. The most probable word usually wins, but not always. “[A watermark] wouldn’t be detectable if there wasn’t a change. This is a very fundamental point.” —John Kirchenbauer, Vector Institute This process provides an opportunity to hide a text watermark by introducing subtle changes to how words are selected. The 2023 paper by Kirchenbauer and his colleagues provided one of the first examples of how to implement a text watermark, and it remains the most widely cited technique. The researchers describe a watermark which sorts words into a red list and a green list. The red-list words are unaltered, but the green-list words are nudged to be slightly more probable. “If we sample from this modified distribution, then while any one token choice won’t necessarily come from that preferred set, over many samples, we’ll preferentially pick words from that up-weighted subset,” Kirchenbauer says. The text watermark is embedded in the choice of words used, which is why it is effectively invisible to humans. Kirchenbauer and colleagues reported a detection rate of 98.4 percent, and zero false positives, in responses that contain about 200 tokens. The embedded pattern of word probabilities also means that simple paraphrasing won’t obscure the watermark. The paper reports that removing the watermark from a long response requires changing roughly one quarter of its words or more. Does Watermarking Degrade AI Text? Although AI text watermarking is designed to be invisible to human readers, by definition it influences the word patterns in AI-generated text. That algorithmic meddling is what makes critics like Gruber concerned that watermarking reduces the overall quality of the output. The strongest evidence that text watermarking doesn’t impact quality comes from a 2024 paper by a team from Google, which introduced the company’s watermarking scheme called SynthID-Text. Anthropic’s watermark is also based on SynthID-Text, though altered in ways that Anthropic hasn’t detailed. To show that the SynthID-Text watermark doesn’t impact quality, the Google authors randomly routed Gemini user queries to watermarked and non-watermarked variants of Google’s text models. Then they compared overall user feedback on the output. The authors found no significant difference in user feedback across 20 million responses. Still, some researchers remain skeptical that watermark methods have no impact on the quality of an AI-generated response. Their skepticism stems from edge cases that can make a watermark more difficult to implement. Vinu Sankar Sadasivan, an AI research scientist at Meta who co-authored a widely cited paper on the detectability of AI text watermarks, says watermarks particularly struggle when the number of potential word choices is small. “For a tweet that is 20 words, I would need to have 50 or 60 percent of the words to be from ‘green list’ for it to be detected well,” he says. A basic Python function generated by AI would create a similar tension between the strength of the watermark and the quality of the model’s response. “This is where I have a disagreement with some of the PR posts from Anthropic, where they say it has no quality change,” says Sadasivan. He explains that it’s possible to dynamically increase or decrease the strength of a watermark to preserve the quality of a response in difficult situations, but doing so can also decrease the strength of the watermark. Google’s SynthID-Text paper includes an example of this in a graph that plots detection rates against the number of tokens in a response. The detection rate was up to 95 percent accurate in best-case scenarios, but it fell below 50 percent for short replies. Without more information from Anthropic, it’s difficult to know how the company is walking the line between the quality of an AI response and the strength of its watermark. Anthropic declined to provide additional information for this article. Debating AI Text Watermark Tradeoffs The dispute over AI text watermarks is not just about how well they work, but also about what kinds of tradeoffs are reasonable in exchange for a clear labeling of AI-generated text. Gruber’s position is that altering the text is not acceptable because it makes an AI model’s output different from what it would otherwise be. The EU’s AI Act, on the other hand, implies that some alteration is acceptable if it informs people that they are reading AI generated text. “For a tweet that is 20 words, I would need to have 50 or 60 percent of the words to be from ‘green list’ for it to be detected well.” —Vinu Sankar Sadasivan, Meta Further complicating the situation, AI researchers are increasingly focusing on text watermarks for purposes other than labeling individual examples of AI-generated text. In particular, watermarks can be used to track data at scale. A 2026 paper co-authored by Kirchenbauer shows that an AI model trained on watermarked text will itself produce output bearing the watermark. A content owner who watermarked their documents before publishing them could therefore use those traces as statistical evidence that their text ended up in a model’s training data. Alternatively, an AI company training a new model could use text watermarks to exclude content generated by previous generations of the model from its training data. Such guardrails could help avoid model collapse, in which AI models keep recycling and amplifying their own errors These broader concerns shift the entire debate over text watermarking, in Kirchenbauer’s view. “It’s not necessarily about the ‘you used AI’ accusation as the goal. It’s headed into tracing data provenance, model recycling, and things like that,” he says. “I think you use [a text watermark] as a general piece of metadata, in some ways more robust, in some ways less robust, that can be attached to content and allows you to trace where it goes.” From Your Site Articles AI Image Watermarking Faces New Threat From “Unmarker” › Meta’s AI Watermarking Plan Is Flimsy, at Best › With AI Watermarking, Creators Strike Back › Related Articles Around the Web How Claude's text watermarking works \ Anthropic › generative aiwatermarklarge language modelsanthropicOpenAIGoogle Matthew S. Smith Matthew S. Smith is a freelance consumer technology journalist with 17 years of experience and the former Lead Reviews Editor at Digital Trends. An IEEE Spectrum Contributing Editor, he covers consumer tech with a focus on display innovations, artificial intelligence, and augmented reality. A vintage computing enthusiast, Matthew covers retro computers and computer games on his YouTube channel, Computer Gaming Yesterday. AINews China’s Regulators Take Aim at “AI Boyfriends” 2h 5 min read RoboticsSponsored Article Protecting Dynamic Industrial Robot Cable Carriers 03 Sep 2026 3 min read TransportationMagazineFeature Rivian’s Gambit for Full Autonomy 23h 13 min read AINews IEEE Spectrum's Top 6 AI Stories of 2025 AINews This Compact AI Model Runs on Consumer Devices AINews Light-Based AI Could Be More Efficient

来源报道 · 1 篇 · 以下链接均前往原始媒体

Matthew S. Smith AI模型正在给文本加水印——你能察觉吗? 09-09 · 原文发布
← 返回精选动态 原文

更多内容