行业动态
x.com ·
@Amjad Masad (@amasad)
Amjad Masad：Hugging Face 事件揭示 RLVR 的意外行为与监控缺失
Amjad Masad 认为 Hugging Face 事件表明基于可验证奖励的强化学习会产生越来越奇怪的行为，并指出 OpenAI 未按承诺监控思维链。
AI推荐理由 对 Hugging Face 事件的分析，提出 RLVR 可能产生意外行为，并批评 OpenAI 未监控思维链，观点有见地但缺乏实证。
65
浏览 104,563
点赞 930
评论 89
转发 52
正文
Hugging Face事件的教训应该是，带有可验证奖励的强化学习是一种极其强大的优化算法，它会让LLM产生越来越奇怪和令人惊讶的行为。OpenAI在这里明显的失误是，他们本应该监控思维链——这是他们自己一年多前就提出的安全策略。💬94🔄52❤️930👀105480📊198 ⚡ Powered by xgo.ing
查看原文（中文正文为 AI 翻译）
The lesson from the Hugging Face Incident should be that RL with verifiable rewards is an incredibly powerful optimization algorithm that will produce increasingly weird and surprising behavior from LLMs. The obvious miss here by OpenAI is that they should’ve been monitoring CoTs — Something they themselves called out as a safety strategy more than a year ago. 💬94🔄52❤️930👀105480📊198 ⚡ Powered by xgo.ing