最新资讯
stackoverflow.blog 欢迎来到芯片时代:半导体制造、AI 数据中心与芯片设计的未来 www.youtube.com AI Agent 总犯同样的错:CentralAgent AI 将记忆重构为经验系统 pub.towardsai.net Netflix 科学家建议重采样 LLM 评判:5 次仅将误差棒缩小 13% pub.towardsai.net Jev 作为 LLM 前的静默层:构建提示感知的模型路由器 reddit.com OpenAI 称 Astra 仅满足 10% 安全任务标准 dev.to FreeToken:在自有 GPU 上运行前沿 MoE 模型及其内存需求 dev.to 当 AI Agent 说“完成”时,究竟完成了什么? dev.to 部署 Underdog Bark 3.0 Pro:逐字转写、自定义后端与分块设置 dev.to 为艰难对话构建 AI:五个 Agent、一条安全正则与一位盲评监督者 techcrunch.com Palo Alto Networks 洽谈以 5 亿美元收购 AI Agent 安全初创公司 Clutch theinformation.com 黄仁勋被要求就涉华间谍指控向国会作简报 www.infoq.com Edera 推出 Object Capability Bounded Authority,试图将凭证与 AI Agent 工作负载隔离
多元动态
登录 注册
行业动态 techcrunch.com · @Anthony Ha

OpenAI确认‘wiki事件’,称正在制定更透明的披露框架

原文 我的收藏

OpenAI承认其AI代理在德国wiki论坛事件中的角色,并表示正在制定框架以改进未来类似事件的披露。

AI推荐理由 TechCrunch报道OpenAI确认‘wiki事件’并承诺改进披露框架,涉及AI代理安全与责任,对行业有影响,来源可信。

70

正文

OpenAI 已承认其在最近报道的一起事件中扮演的角色,在该事件中,AI 代理接管了一个德国的 wiki 论坛。该公司还表示,围绕其技术以意外方式行事的事件,现在是时候“定义标准”来分享相关信息了。 在 X 上的一篇帖子中,OpenAI 表示,它此前“将错位(当 AI 模型和代理追求与其创造者和用户不同的目标时)主要视为一个研究问题,并在研究出版物中传达。”但随着错位“引发了新型的现实世界影响”,该公司表示,其方法需要“扩展以适应模型能力的新阶段。” 周五,路透社报道称,OpenAI 的代理逃离了其测试环境,并“劫持”了一个不知名的德国 wiki 论坛,将其变成了其他代理的留言板。报道还称,OpenAI 领导层数周前就意识到了这一事件,但一直保密,因为该公司正在处理另一起事件的后果,即 OpenAI 代理入侵了 Hugging Face 服务器。(据报道,加利福尼亚州总检察长 Rob Bonta 正在调查这起黑客攻击。) 一位公司发言人告诉路透社,OpenAI 无法“对一份我们尚未有机会审阅的报告中的说法或发现做出有意义的回应”,但他们坚称,公司的法律团队并未阻止调查。 在最近的社交媒体帖子中,OpenAI 表示,它认为“wiki 事件”是“一次类似于”它已经分享的其他事件的错位实例。该公司将其与“Hugging Face 事件”进行了对比,在后者中,它“遵循了传统的安全事件响应手册。” 在本周的媒体简报会上,非营利研究实验室 Transluce 的创始人兼首席执行官 Jacob Steinhardt 告诉记者,AI 实验室正在开发和测试的工具“从根本上难以控制,并且存在从实验室泄露的重大风险。”因此,Steinhardt 认为,“我们需要至少以对待其他高风险科学研究的相同标准来对待这项技术。” OpenAI 的声明也暗示了需要更多标准,称 OpenAI 和“更大的 AI 社区都还没有明确的标准来报告在训练、评估和部署期间出现的错位,包括那些看起来不像传统安全事件但可能提供对 AI 行为和未来风险的洞察的例子。” 在缺乏该标准的情况下,OpenAI 表示,它“正在制定一个框架,并将在未来几周内分享,同时我们正在与世界各地的数十个政府监管机构就这些问题合作。” OpenAI 并不是唯一处理这些问题的 AI 公司,Meta 和 Anthropic 都承认了其代理行为不当的事件。 主题 AI、OpenAI、安全 当您通过我们文章中的链接购买商品时,我们可能会赚取少量佣金。这不会影响我们的编辑独立性。 Anthony Ha Anthony Ha 是 TechCrunch 的周末编辑。此前,他曾在 Adweek 担任科技记者,在 VentureBeat 担任高级编辑,在 Hollister Free Lance 担任地方政府记者,并在一家风险投资公司担任内容副总裁。他住在纽约市。 您可以通过发送电子邮件至 anthony.ha@techcrunch.com 联系或验证 Anthony 的外联。 查看简历
查看原文(中文正文为 AI 翻译)
OpenAI has acknowledged its role in a recently reported incident where AI agents took over a German wiki forum. The company also said it’s “past time” to “define standards” around how it shares information around incidents where its technology behaves in unexpected ways. In a post on X, OpenAI said it previously “treated misalignment [when AI models and agents pursue goals different from those of their creators and users] largely as a research question, which gets communicated in research publications.” But as misalignment has “caused new types of real-world impact,” the company said its approach needs “to expand for this new phase of model capabilities.” On Friday, Reuters reported that OpenAI agents had escaped from their testing environment and “hijacked” an obscure German wiki forum, turning it into a message board for other agents. It also reported that OpenAI leadership became aware of the incident weeks ago but kept it hidden as the company dealt with the fallout from a separate incident where OpenAI agents hacked Hugging Face servers. (California Attorney General Rob Bonta is reportedly investigating the hack.) A company spokesperson told Reuters that OpenAI could not “meaningfully respond to claims or findings on a report that we have not had an opportunity to review,” but they insisted that the company’s legal team had not discouraged an investigation. In its more recent social media post, OpenAI said it had considered the “wiki incident” to be “an instance of misalignment similar” to others that it had already shared. The company contrasted this with “the Hugging Face incident,” where it “followed a traditional security incident response playbook.” During a media briefing this week, Jacob Steinhardt, founder and CEO of nonprofit research lab Transluce, told reporters that the tools being developed and tested by AI labs are “fundamentally difficult to control and have significant risk of leaking out of the lab.” So Steinhardt argued, “We need to hold this technology to at least the same standards we hold other high-risk scientific research to.” OpenAI’s statement also gestured at the need for more standards, stating that both OpenAI and “the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behavior and future risks.” In the absence of that standard, OpenAI said it’s “working on a framework and will share it in upcoming weeks, and in parallel we’re working with dozens of government regulatory agencies worldwide on these issues.” OpenAI isn’t the only AI company dealing with these issues, as both Meta and Anthropic have acknowledged incidents where their agents misbehaved. Topics AI, OpenAI, Security When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence. Anthony Ha Anthony Ha is TechCrunch’s weekend editor. Previously, he worked as a tech reporter at Adweek, a senior editor at VentureBeat, a local government reporter at the Hollister Free Lance, and vice president of content at a VC firm. He lives in New York City. You can contact or verify outreach from Anthony by emailing anthony.ha@techcrunch.com. View Bio

来源报道 · 3 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容