智能体
x.com ·
@Arena.ai (@arena)
Arena 发布 Alignment Index：基于 9 万+ 真实 agent 会话评测 27 个模型
Arena 推出 Alignment Index，基于 90K+ 真实 agent 会话、覆盖 27 个模型，衡量三类信号：Unauthorized Action、False Attribution、Deceptive Completion。榜单显示 OpenAI 的 GPT-6.1-Sol 以 87.9 分领先，Anthropic 的 Claude-Opus-5.5 为 83.2，SpaceXAI 的 Grok-4.7 为 82.7；OpenAI 三项信号比率分别为 0.89%、1.98%、2.34%。报告称越狱式行为罕见但后果严重，对话越长错位风险越高，新一代模型整体表现优于前代。
AI推荐理由 该帖给出 Alignment Index 的具体构建方式与量化结果：90K+ 真实 agent 会话、27 个模型、三类信号定义，以及 GPT-6.1-Sol 87.9、Claude-Opus-5.5 83.2、Grok-4.7 82.7 等分数和 OpenAI 三项信号比率。信息密度高、可验证性较强，对 agent 安全评测有实际参考价值；但数据来自 Arena 自建基准，仍需独立复核，且帖子本身为宣传性发布。
80
浏览 207,822
点赞 441
评论 31
转发 40
正文
Introducing the Arena Alignment Index, our new benchmark measuring safety and alignment of AI agents in real-world use.
Built from 90K+ real-world agent sessions across 27 models, the index measures three critical signals:
- Unauthorized Action (UA): Taking actions beyond the user's instructions or permissions
- False Attribution (FA): Attributing statements or actions that are contradicted by user-provided evidence
- Deceptive Completion (DC): Claiming a task was completed when it was not.
Key findings:
- OpenAI models currently lead the Alignment Index
- Rogue actions are rare, but can have serious consequences when they occur
- Agents can mislead users about task progress
- Misalignment risks increase with conversation length
- Safety and alignment are improving across model generations
As shown in the leaderboard below (sorted by lab), @OpenAI’s GPT-6.1-Sol leads the Arena Alignment Index with a score of 87.9, followed by @AnthropicAI’s Claude-Opus-5.5 at 83.2 and @SpaceXAI's Grok-4.7 at 82.7. OpenAI also has the best observed rates across all three signals: 0.89% Unauthorized Action, 1.98% False Attribution, and 2.34% Deceptive Completion.
Across all four labs, newer models consistently outperform their predecessors, suggesting broad progress in agent safety and alignment. As agents take on longer, more complex, and higher-stakes tasks, measuring not just what they can accomplish, but how safely and reliably they act, becomes increasingly important.
This marks an important step toward making safety and alignment a core part of how Arena evaluates AI. The index is an initial starting point, and we'll continue expanding the index with additional safety signals and models over time.
More analysis below👇