大模型
x.com ·
@akothari
Databricks 内部评测：Opus 5.5 与 GPT-6 Luna 推进成本/质量前沿
Databricks 的 Patrick Wendell 分享基于 2,400 名工程师在线负载与离线评测的发现：上周发布的三款模型中，Opus 5.5 与 GPT-6 Luna 明显扩展成本/质量前沿。Opus 5.5 成为最高质量的中端模型，优于此前所有 Opus 及 GPT-6 Sol、GPT-5.6 Sol，同任务成本较 Opus 4.8 稳定降低约 20%，被视为编码日常默认模型的候选。GPT-6 Luna 每任务成本至少比 Opus 5.5 低 20 倍，在某高难评测上大致追平 Opus 4.6，而每任务成本低 99.3%，但该结论为初步结果，仍在更广泛评测中。
AI推荐理由 Databricks 员工转述的内部评测结论，给出 N=2,400 工程师在线负载与离线评测的具体对比：Opus 5.5 在同任务成本上较 Opus 4.8 降低约 20%，GPT-6 Luna 每任务成本至少便宜 20 倍，并在某高难评测上接近 Opus 4.6 而成本低 99.3%。信息密度高、有量化口径，但属二手转述且部分结论（Luna 质量）作者自述仍属初步、待更广评测，故未给更高分。
72
浏览 150
点赞 0
评论 0
转发 0
正文
[x.com/pwendell/statu…](https://x.com/pwendell/status/2104643192209747979)
Patrick Wendell@pwendell
Crazy few weeks for model releases! Our findings [@Databricks](https://x.com/Databricks) show several new models meaningfully advance the pareto frontier. Results below (online workload analysis of N=2,400 engineers, plus offline evals):
1. Two of the three models released last week clearly expand the cost/quality frontier: Opus 5.5 and GPT-6 Luna.
2. Opus 5.5 is now the highest quality mid-tier model. It is better than all prior Opus models, better than GPT-6 Sol, and better than GPT-5.6 Sol.
3. Opus 5.5 reduces same-task costs consistently by 20% in both offline and online analysis. This is against a baseline of Opus 4.8, the prior least-cost Opus model (Opus 5.0 was a bit of a dud with high costs and barely noticeable quality improvements).
4. Due to best-in-class quality and lower costs, Opus 5.5 is a strong candidate as an “every day default” model for coding, and we are now encouraging it for this purpose at Databricks.
5. GPT-6 Luna is very, very, very cheap. It was at least 20 times cheaper per-task than Opus 5.5 in every offline benchmark we tested and in observed online use.
6. GPT-6 Luna is surprisingly capable given how cheap it is. On one of our most difficult evaluation suites it roughly matches Opus 4.6 performance, while being 99.3% cheaper per-task than Opus 4.6 was at that time. That's a 100X cost reduction in ~9 months! This finding is preliminary and we are still evaluating Luna quality on a broader set of offline and online tests.
Our production setup: Unity Gateway to route workloads across models and trace agentic interactions. A mix of end-user harnesses including: Omingent (meta-harness), Claude Code, Codex, and Cursor.
[🔗 View Quoted Tweet](https://x.com/pwendell/status/2104643192209747979)