开源工具
x.com ·
@sama
Sam Altman：前沿 AI 需要一致安全规则，OpenAI 在 frontier RL 训练前预先制定安全案例
Sam Altman 发帖称，世界应相信开发日益强大 AI 的美国公司会负责任行事，每家前沿实验室都必须做到。他欢迎为前沿 AI 设定一致安全要求的联邦框架，但认为不必等待反垄断豁免或立法即可开始工作，并对独立审计等思路表示期待。他指出，Responsible Scaling Policies 和 Preparedness Frameworks 主要聚焦已发布模型的部署，而非开发过程；如今转向安全开发与评估需要新工具，例如 OpenAI 现在会在预期显著提升能力的前沿强化学习训练前预先制定明确的安全案例。他强调'pacing'不是'停止'，安全干预有显著成本但值得，国际协调需要政府协助。
AI推荐理由 Sam Altman 关于前沿 AI 安全治理的完整表态，包含多项具体信息：欢迎联邦统一安全要求框架、支持独立审计、指出 Responsible Scaling Policies 与 Preparedness Frameworks 主要针对已发布模型而非开发过程、OpenAI 在 frontier RL 训练前预先制定安全案例、'pacing 不等于 stopping'、以及国际协调需要政府协助。属于一线实验室负责人对安全政策与内部流程的公开说明，对理解前沿 AI 治理走向有实际价值。
80
浏览 2,856,250
点赞 10,369
评论 1,952
转发 912
正文
The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.
We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors).
Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process.
Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases.
We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these.
When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.
Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.
Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.