最新资讯
dev.to 你的 Agent Harness 应具备的 17 项能力 dev.to 离线 AI 记忆中 0.8 余弦阈值为何失效 dev.to Kimi K3.1 Pro 开放权重:自托管 2.8T 模型前的开发者清单 dev.to 如何通过工具调用引导模型以提升 Harness 质量 dev.to AI Agent 审计轨迹不等于对话记录 dev.to 精简 MCP server 工具数量:更少工具、更小影响面,双重收益 dev.to 为什么 git checkout 无法撤销 AI 编码 agent 的改动? wechat AI短剧门槛降降降!实测即梦短剧工作室:剧本分镜预演一条龙,伙伴计划长期护航 dev.to 让模型改写文档前,先量化事实保真度 dev.to 回放 153 个 Claude Code 与 Codex 会话,审计 Agent 的生产影响范围 dev.to 银行 77 意图分类最便宜 LLM:GPT-6.1 Terra 每次成功任务 $0.000727 theinformation.com Chesky的副业:Airbnb向CEO新设AI实验室出售IP
多元动态
登录 注册
开源工具 dev.to · @CacheLine

写代码 10 年真正改变的是什么(不是代码本身)

原文 我的收藏

文章引用 METR 于 2025 年开展的随机对照试验:研究者让 16 名经验丰富的开源开发者在自己已非常熟悉、积累多年上下文的开源仓库中处理真实 issue,并随机分配是否使用 AI 工具。开发者事前预计 AI 可将完成时间缩短约 24%,完成后仍认为大约快了 20%,但秒表显示他们实际比不用 AI 时慢 19%,形成约 39 个百分点的差距。该研究强调在真实成熟代码库而非基准题上测试 AI 的实际表现。

AI推荐理由 文章引用 METR 2025 年随机对照试验:16 名经验丰富的开源开发者在自己熟悉的成熟代码库上完成任务,事前预计 AI 可提速约 24%,事后仍认为约快 20%,但实际用时比不用 AI 慢 19%,形成约 39 个百分点的认知偏差。该结果对评估 AI 编码工具的真实生产力有直接价值,且来源为受控实验而非厂商宣传;但摘要未给出样本量之外的统计细节与任务类型分布,需注意结论适用范围。

68

正文

Ask sixteen experienced open-source developers how much faster AI tools would make them, and they'll say about 24%. Ask them again right after they finish the task, and they'll still swear they were roughly 20% faster. The stopwatch says otherwise: they were actually 19% slower. That gap is the headline result from a 2025 randomized controlled trial run by METR, a group that studies how AI systems actually perform in the wild rather than on benchmarks. Instead of testing beginners on toy problems, the study put experienced developers to work on real, mature codebases they already knew well — their own open-source repositories, with years of accumulated context already living in their heads. The 39-point gap Here's the setup: developers picked real issues from their own projects, then were randomly assigned to either use AI tools or not for each task. Before starting, they predicted AI would cut their completion time by about 24%. After finishing, they still believed they had come out roughly 20% ahead. The actual clock time told a different story. Tasks done with AI assistance took about 19% longer than tasks done without it. That's a 39 percentage-point gap between what these developers believed happened and what a stopwatch recorded — in professionals who write code for a living. Why the most experienced developers lost the most time The slowdown wasn't evenly spread. It was strongest on exactly the codebases the developers knew best. That's the counterintuitive part: you'd expect deep familiarity to make AI assistance additive, a second pair of hands on a system you already understand. Instead, the tool mostly added overhead — prompting, reviewing suggestions, and correcting output — layered on top of work the developer could already do quickly from memory. Ten years of shipping code builds a fast, private, mental model of a system: which functions are safe to touch, which abstractions leak, where the real bugs hide. That model runs in the background and is basically free. An AI assistant doesn't have it, so every suggestion has to be checked against it, and checking takes time the raw "assistance" never shows up as. The same illusion shows up in floating point There's a smaller, more mechanical version of this mismatch that any developer runs into on day one: >>> 0.1 + 0.2 0.30000000000000004 Enter fullscreen mode Exit fullscreen mode Nothing is broken. Binary floating point can't represent 0.1 or 0.2 exactly, so the addition is off by a sliver too small to notice by eye but large enough to break an equality check. The computer isn't wrong — your intuition about what "0.1 + 0.2" means is just miscalibrated against how the machine actually stores numbers. That's the same shape as the METR result. Your internal sense of speed, like your internal sense of what 0.1 plus 0.2 equals, is a model — usually good enough, occasionally confidently wrong, and only correctable by actually checking the value against a ground truth instead of trusting the feeling. What actually worked The METR study also found the developers who avoided the slowdown weren't the ones who resisted AI tools altogether. They were the ones who stopped guessing and started timing themselves — treating "does this actually help me" as a measurable question instead of a vibe. The other habit that held up: writing down why the code looks the way it does, before that reasoning quietly leaves with whoever holds it in their head. A decade of experience is mostly an unwritten, unmeasured model of a system. AI tools are good at producing plausible-looking code fast; they are not a substitute for that model, and treating them as one is exactly how an experienced developer ends up slower while feeling faster. If you want to know whether a tool is actually helping you, the fix is unglamorous: time it, on your own real tasks, more than once. Here's the 60-second version of this story: If you've caught yourself feeling faster while a task actually dragged on, that's the whole study in miniature — worth checking with a clock next time before you trust the feeling.

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容