智能体
pub.towardsai.net ·
@allglenn
Grok 4.7 看似突破，直到你检查 effort level
文章讨论 xAI 于 2026 年 9 月 21 日发布的 Grok 4.7，指出其发布记分卡可能因 effort level、agent harness 和 evaluation operator 三个隐藏变量而失真。作者认为基准分数是配置化系统的结果，不应直接作为生产迁移或采购依据，并称这是所有前沿实验室发布图表的共性问题。
AI推荐理由 文章以 xAI 2026 年 9 月 21 日 Grok 4.7 发布为背景，指出发布记分卡中 effort level、agent harness、evaluation operator 三个隐藏变量可能使对比结果失真，提醒不要把基准分数当作采购依据。这是对基准评测方法论的合理质疑，但正文片段仅给出论点框架，未提供具体数据或复现证据，且为 Medium 会员文章，信息密度有限。
62
正文
Member-only story
Grok
LLM
Artificial Intelligence
Coding
Technology
Grok 4.7 Looks Like a Breakthrough Until You Check the Effort Level
One scorecard, three hidden variables.
allglenn
8 min read
·
Just now
--
Listen
Share
Press enter or click to view image in full size
Grok 4.7
In its 21 September 2026 launch announcement, xAI positioned Grok 4.7 as an upgrade for coding and knowledge work at the same list price and context class as Grok 4.6.
The headline numbers may be accurate. They can still be a poor basis for a production migration.
The issue is not whether xAI fabricated a benchmark. It is whether the compared columns use the same effort level, agent harness, and evaluation operator. Those variables can change a result as much as the model revision itself.
This is not unique to xAI. Every frontier lab has a version of the same problem: a launch chart compresses a complicated evaluation protocol into a clean story. Treat it as a claim to investigate, not a procurement document.
A benchmark score is a configured system
A launch scorecard resembles a pricing page. The rates can be real while the conditions that determine the final bill sit outside the headline.
For agent benchmarks, a score is often a result from a configured system:
Effort: low, medium, high, xhigh, or a vendor-specific equivalent.