论文研究
x.com ·
@lmarena.ai(@lmarena_ai)
腾讯混元发布 WebCraftBench：用真实用户请求评测 AI 生成应用
LMArena 介绍腾讯混元团队的 WebCraftBench，该基准由 369 条来自内部 Code Arena 复刻环境的真实请求构成，包含非正式语言、不完整指令和多样偏好，用于评测 AI 构建应用的外观、功能与需求满足度。在 17 个模型上的排名与 Code Arena 榜单相关性为 0.89；引用帖称在 197 对人工验证样本上与人偏好一致率 85.3%，并附有 arXiv 论文。
AI推荐理由 LMArena 官方账号介绍腾讯混元团队的 WebCraftBench：基于 369 条来自内部 Code Arena 复刻环境的真实请求构建，覆盖非正式表达、不完整指令等真实使用场景，并在 17 个模型上评测 AI 生成应用的外观、功能与需求满足度，排名与 Code Arena 榜单相关性 0.89。引用帖补充称在 197 对人工验证样本上与人偏好一致率 85.3%，并给出 arXiv 论文编号。信息密度高、有可验证来源，但数据来自转述，需以论文为准。
78
正文
Real-world app requests don’t come as perfect specs.
Congrats to the [@TencentHunyuan](https://x.com/TencentHunyuan) team on WebCraftBench!
Built from 369 requests collected from their internal replica of Code Arena, it embraces the messy details of real human usage: informal language, incomplete instructions, different project scopes, and diverse preferences.
Their benchmark tests how AI-built apps look, work, and deliver on those requests. Across 17 models, its rankings strongly correlate with our Code Arena leaderboard (0.89).
What could you uncover from real human-AI conversations and Arena’s leaderboard history? Explore our open datasets at [huggingface.co/lmarena-ai/dat…](http://huggingface.co/lmarena-ai/datasets)!
Tencent Hy@TencentHunyuan
The AI says the site is done. Then the homepage errors, the buttons overlap, and it added a login flow you never asked for. 🙃
Introduces WebCraftBench: agents actually use the live app, coverage-guided exploration finds what never got reached, then we score aesthetics, usability, and whether the original request was met.
On 197 human-validated pairs, it matches human preference 85.3% of the time.
Paper:[arxiv.org/abs/2609.15387](https://arxiv.org/abs/2609.15387)Z
[🔗 View Quoted Tweet](https://x.com/TencentHunyuan/status/2102320407860977861)
💬0🔄0❤️1👀1237