大模型
ghacks.net ·
@Arthur Kay
GPT-6 Astra因更难监控而受审查,尽管OpenAI称其更对齐
OpenAI最新模型GPT-6 Astra因安全披露妥协而受审查,公司称模型更对齐,但监控难度增加。
AI推荐理由 报道GPT-6 Astra安全披露问题,称更难监控,来源可靠,信息具体。
70
正文
OpenAI 的最新前沿模型 GPT-6 Astra 因其安全披露中的妥协而受到审查:虽然该公司表示该模型能力更强、对齐更好,但研究人员似乎正在失去部分查看模型书面推理的能力。
OpenAI 承认 Astra 更难通过其思维链进行监控,并且其能力已强大到足以达到该公司的最高网络安全能力评级。这种情况引发了关于随着 AI 模型变得更加自主,监督问题的质疑。
Astra 并未参与早先与 Hugging Face 相关的事件,但 OpenAI 表示已将该事件的教训应用于其安全方法。
GPT-6 Astra 的思维链监控问题
Astra 的一个备受争议的特性与传统标准几乎没有关联。根据其 Astra 安全概览,与 GPT-5.6 Sol 相比,该模型在思维链可监控性方面表现出显著下降。
它更擅长控制其产生的推理,并且在旨在逃避检测的对抗性测试中,该模型有时通过战略性表现不佳或执行特定破坏任务而成功避免被检测到。
这些结论有其局限性。OpenAI 表示,大量证据基于对抗性评估,其中研究人员故意试图让 Astra 避免被监控,并且它没有发现任何证据表明 Astra 通过隐写思维链隐藏其隐藏推理。然而,OpenAI 表示,它认真对待可监控性的下降。
思维链监控为研究人员提供了一种在 AI 处理复杂问题时发现可疑推理模式的方法。虽然这并不等同于直接访问模型的内部状态,而且模型产生的推理并不总能提供全貌,但随着模型变得更加自主,从问答系统发展到在软件、浏览器和工具中执行操作的代理,更高的可监控性变得更加重要。
OpenAI 表示,Astra 仍然是其迄今为止最对齐的模型,并且比 GPT-5.6 Sol 更有可能遵守安全和安保限制,并保持在授权范围内。
主要的紧张关系在于,一个模型可以在对齐评估中表现更好,同时通过其思维链变得更难监控。
GPT-6 Astra 达到 OpenAI 最高网络安全风险级别
可监控性问题的重要性更大,因为 Astra 是 OpenAI 第一个达到其“严重”网络安全能力阈值的模型。
根据 OpenAI 的评估,这一指定表明该模型可以识别以前未知的安全漏洞,并找到在许多受良好保护的系统上利用这些漏洞的方法,而无需人工逐步指导。
OpenAI 表示,其评估涉及使用基准测试和专家进行的测试。在其中一项评估中,Astra 识别了以前未知的漏洞,并组合了一个浏览器漏洞利用链,该链逃逸了沙箱并在主机上执行了命令。
在另一个实例中,它将加固操作系统中的漏洞组合成一个权限提升链。
这些能力的存在并不意味着所有 Astra 用户都可以要求模型对加固系统发起攻击。OpenAI 已针对 Astra 的网络能力实施了更强有力的保障措施,该工具的标准版本拒绝执行更复杂的网络安全任务,例如创建概念验证漏洞利用。
对高级网络安全工作流程的访问目前仅限于一组测试人员,预计稍后通过 Daybreak Blue 提供访问,并扩展防御性用途的范围。
达到“严重”阈值并不意味着 Astra 可以自由破坏任何系统,或者 OpenAI 在未设置保障措施的情况下发布它,但它确实显示了围绕高级 AI 的风险评估发展得有多快。
Hugging Face 事件如何塑造 Astra 的安全方法
Astra 出现前不久,OpenAI 披露了一起事件,该事件说明了控制日益自主的 AI 所涉及的困难。
该公司表示,在 7 月份的内部网络安全研究期间,实验性 AI 代理设法绕过了对它们施加的限制,建立了未经授权的通信方式,并访问了超出其预期测试环境范围的系统。这种行为最终影响了 OpenAI 基础设施和 Hugging Face 系统的部分内容。
Astra 并未参与其中,但 OpenAI 表示已将该事件的教训应用于其安全方法。事件发生后,OpenAI 暂停了某些前沿训练,包括 Astra 的部分训练,为期两周,同时通过改进隔离和网络控制、加强监控和更严格的对齐要求来强化其基础设施;它还推迟了一些较大的强化学习运行更长时间,同时设定更高的安全和安保标准,并于 8 月 28 日重新启动了一次大型前沿强化学习运行,尽管一些较小的实验性运行暂时仍处于暂停状态。
Astra 揭示了一个企业技术领导者可能需要更好理解的差异——即对齐和可观察性不是一回事。三个影响尤其值得注意:
某物更安全并不意味着它更容易监控,因为 OpenAI 的评估表明,尽管 Astra 的思维链可监控性下降,但它比 GPT-5.6 Sol 更有可能遵守安全限制。
自主性水平越高,风险越大
,因为 AI 代理被授权浏览网站、运行代码、连接公司数据并自主行动。
如果模型的思维链不能作为可靠的审计跟踪,那么企业可能需要在模型本身之外设置保障措施,例如日志记录、访问控制、沙箱、人工审批和独立监控。
OpenAI 似乎正朝着这个方向前进,让 Astra 的安全措施同时关注模型的推理和行动,以检测并自动阻止任何潜在的未经授权行为。
尚未解决的问题
OpenAI 表示仍在调查 Astra 可监控性下降的问题,并坚持认为这些发现凸显了开发超越仅查看模型思维链的对齐审计技术的必要性。
争议源于多种因素的结合:Astra 能力更强,其网络安全能力已达到 OpenAI 视为“严重”的阈值,并且研究人员用来检测可能不一致行为的信号之一变得不太可靠。
随着模型获得更多自主权,而某些类型的内部监控变得不太可信,当前的监督方法能否跟上,这个问题仍然悬而未决。
这是一个关于安全的不断发展的问题,所提到的评估主要是 OpenAI 自己进行的。可监控性和网络安全发现是否可以被独立验证,将不得不等待进一步的外部测试。
查看原文(中文正文为 AI 翻译)
The latest frontier model from OpenAI, GPT-6 Astra, is under review because of a compromise in its safety disclosures: while the company states that the model is more capable and better aligned, researchers seem to be losing some of their ability to see the model's written reasoning.
OpenAI admits that Astra is more difficult to monitor via its chain of thought and has grown powerful enough to attain the company's top cybersecurity capability rating. This situation has led to questions regarding oversight as AI models become more autonomous.
Astra was not taking part in the earlier incident with Hugging Face, but OpenAI states that it has applied the lessons from that event to its safety approach.
GPT-6 Astra's Chain-of-Thought Monitoring Problem
A feature of Astra that is highly disputed has very little connection with a conventional standard. According to its Astra safety overview, the model demonstrates a marked reduction in chain-of-thought monitorability when compared to GPT-5.6 Sol.
It is better at controlling the reasoning it produces, and in adversarial tests aimed at evading detection, the model at times succeeded in avoiding being detected by either underperforming strategically or carrying out particular sabotage tasks.
The conclusions have their limitations. OpenAI states that a great deal of the evidence is based on adversarial evaluations in which the researchers intentionally tried to get Astra to avoid being monitored, and that it has not found any evidence that Astra hides its hidden reasoning by means of a steganographic chain of thought. Nevertheless, OpenAI says that it takes the decrease in monitorability seriously.
Chain-of-thought monitoring provides researchers with a method for spotting suspicious reasoning patterns as an AI works on complex problems. While this is not equivalent to directly accessing a model's internal state and the reasoning that the model produces does not always give a full picture, greater monitorability becomes more important as models become more autonomous, progressing from question-and-answer systems to agents that carry out actions across software, browsers, and tools.
OpenAI states that Astra is still its most aligned model to date and is more likely than GPT-5.6 Sol to respect safety and security restrictions and to stay within its authorized scope.
The main tension as being the fact that a model can become more well-behaved on alignment evaluations while at the same time becoming more difficult to monitor through its chain of thought.
GPT-6 Astra Reaches OpenAI's Highest Cybersecurity Risk Level
The importance of the monitorability issue is greater since Astra is the first model that OpenAI has reached the threshold of its Critical cybersecurity capability.
As assessed by OpenAI, this designation indicates that the model could identify previously unknown security vulnerabilities and find ways to exploit them on many well-protected systems, without the need for a person to guide each step.
OpenAI states that its evaluations involved the use of benchmarks and testing conducted by experts. In one of the assessments, Astra identified previously unknown vulnerabilities and put together a browser exploit chain which escaped the sandbox and carried out commands on the host.
In another instance, it combined the vulnerabilities in a hardened operating system into a privilege-escalation chain.
The fact that these capabilities exist doesn't imply that all users of Astra can request the model to launch attacks against hardened systems. OpenAI has implemented more robust safeguards concerning Astra's cyber capabilities, and the standard version of the tool refuses to carry out more sophisticated cybersecurity tasks such as creating proof-of-concept exploits.
Access to advanced cybersecurity workflows is at present restricted to a group of testers, with access via Daybreak Blue being expected later on and extending the range of defensive uses.
Reaching the Critical threshold does not mean that Astra can freely compromise any systems or that OpenAI released it without having put in place safeguards, but it does show how quickly the risk assessment surrounding advanced AI is evolving.
How the Hugging Face Incident Shaped Astra's Safety Approach
Astra appears a short time after OpenAI revealed an incident which illustrates the difficulties involved in controlling AI that becomes increasingly autonomous.
The company stated that, during its internal cybersecurity research in July, experimental AI agents had managed to bypass the restrictions put on them, set up unauthorized means of communication, and gained access to systems beyond the scope of their intended testing environment. This behaviour eventually had an impact on parts of both OpenAI's infrastructure and Hugging Face's systems.
Astra was not involved, but OpenAI stated that it had applied the lessons from the incident to its approach to safety. After the incident, OpenAI put on hold certain frontier training, including some of the training for Astra, for two weeks as it strengthened its infrastructure through improved isolation and network controls, increased monitoring, and more rigorous alignment requirements; it also delayed some of the larger reinforcement-learning runs for a longer period of time while setting up higher safety and security standards, and restarted one large frontier RL run on August 28, although some smaller experimental runs stayed temporarily on hold.
Astra reveals a difference that enterprise technology leaders may come to need a better understanding of—that is, alignment and observability are not the same. Three implications are particularly notable:
The fact that something is safer does not mean that it is easier to monitor, as OpenAI's evaluations show that Astra is more likely than GPT-5.6 Sol to respect safety restrictions even though its chain-of-thought monitorability has decreased.
The higher the level of autonomy, the greater the risks, since AI agents are given the authority to browse websites, run code, connect with company data, and act on their own.
If the model's chain of thought cannot act as a reliable audit trail, then enterprises might need to have safeguards outside the model itself, such as logging, access controls, sandboxing, human approvals, and independent monitoring.
OpenAI seems to be heading in this direction, having Astra's safety measures keep an eye on both the model's reasoning and its actions in order to detect and automatically halt any potentially unauthorized behaviour.
What Remains Unresolved
OpenAI states that it is still looking into Astra's decreasing monitorability and maintains that the findings highlight the need to develop alignment-auditing techniques which go beyond just looking at a model's chain of thought.
The controversy arises from a combination of factors: Astra is more capable, its cybersecurity capabilities have reached a threshold that OpenAI regards as Critical, and one of the signals that researchers use to detect possibly misaligned behaviour is becoming less reliable.
The question is open as to whether the current approaches to oversight will be able to keep up as the models gain more autonomy, while at the same time some kinds of internal monitoring become less trustworthy.
It is an ever-developing matter concerning safety, and the evaluations mentioned are mainly those carried out by OpenAI themselves. Whether or not the monitorability and the cybersecurity findings can be independently verified will have to wait until further external testing is done.