大模型
website ·
[持续跟踪] Chollet 公布 GPT-6 Astra ARC-AGI-3 评测：连续对话接近满分
François Chollet公布GPT-6 Astra在ARC-AGI-3上的评测结果：标准harness得分66%，连续对话harness接近100%，成本约每游戏360美元。Astra展现出高效的符号世界建模能力，甚至自创DSL。
AI推荐理由 该结果展示模型能力向harness转移的趋势，对理解前沿模型推理能力有重要价值。
79
浏览 31,663
点赞 137
评论 10
转发 13
正文
Marcus，2020年，arXiv：我们需要符号世界模型
几乎所有人（除了[@fChollet](https://x.com/fChollet)）都认为：Marcus错了
今天：GPT-6 Astra通过创建和使用符号世界模型表现得非常好。
开放问题：
1. Astra诱导此类模型的能力有多普遍？
2. 它是如何做到的？
3. 它能可靠地遵循明确的指令吗？
François Chollet@fchollet
GPT-6 Astra代表了交互式推理问题模型能力的阶跃式变化。使用我们的标准测试框架，它在ARC-AGI-3上得分66%，而使用连续对话框架和自定义压缩，得分接近100%，每个游戏成本约为360美元。
事实上，连续框架版本在几乎所有级别上的行动效率都显著优于我们的人类基线。当我们检查推理链以了解模型如何运作时，我们发现它为每个游戏和级别执行了高效、即时的符号世界建模。它甚至开发了自己的速记DSL来表示游戏内情况——本质上是一种特定于游戏的代数符号。
总的来说，Astra表现出我们以前只在复杂框架中见过的符号建模行为——因此框架能力越来越多地转移到模型本身。
我们认为Astra是模型智能的重大突破。
阅读我们关于Astra的帖子以及这些结果的含义：[arcprize.org/blog/astra](https://arcprize.org/blog/astra)
[🔗 查看引用的推文](https://x.com/fchollet/status/2095598451115614371)
查看原文（中文正文为 AI 翻译）
Marcus, 2020, arXiv: We need symbolic, world models
Almost everyone but [@fChollet](https://x.com/fChollet): Marcus is wrong
Today: GPT-6 Astra does very well by creating and using symbolic world models.
Open questions:
1. how general is Astra’s ability to induce such models?
2. how does it manage to do so?
3. can it follow explicit instructions reliably?
François Chollet@fchollet
GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game.
In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation.
Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself.
We see Astra as a major breakthrough in model intelligence.
Read our post on Astra and what these results mean: [arcprize.org/blog/astra](https://arcprize.org/blog/astra)
[🔗 View Quoted Tweet](https://x.com/fchollet/status/2095598451115614371)