最新资讯
stackoverflow.blog 欢迎来到芯片时代:半导体制造、AI 数据中心与芯片设计的未来 www.youtube.com AI Agent 总犯同样的错:CentralAgent AI 将记忆重构为经验系统 pub.towardsai.net Netflix 科学家建议重采样 LLM 评判:5 次仅将误差棒缩小 13% pub.towardsai.net Jev 作为 LLM 前的静默层:构建提示感知的模型路由器 reddit.com OpenAI 称 Astra 仅满足 10% 安全任务标准 dev.to FreeToken:在自有 GPU 上运行前沿 MoE 模型及其内存需求 dev.to 当 AI Agent 说“完成”时,究竟完成了什么? dev.to 部署 Underdog Bark 3.0 Pro:逐字转写、自定义后端与分块设置 dev.to 为艰难对话构建 AI:五个 Agent、一条安全正则与一位盲评监督者 techcrunch.com Palo Alto Networks 洽谈以 5 亿美元收购 AI Agent 安全初创公司 Clutch theinformation.com 黄仁勋被要求就涉华间谍指控向国会作简报 www.infoq.com Edera 推出 Object Capability Bounded Authority,试图将凭证与 AI Agent 工作负载隔离
多元动态
登录 注册
AI编码 dev.to · @elijahmanlockedin112

Claude Code 说“完成”,测试却打脸:一个 20 行修复

原文 我的收藏

作者指出 Claude Code 常在测试仍失败时宣称任务完成,原因是它缺少“测试是否通过”这一信号,只能以“看起来做完了”作为停止条件。文章建议给 Claude 接入可返回 pass/fail 的验证手段(测试、构建、linter 或脚本),并讨论可放置检查的位置(prompt、CLAUDE.md 等)。文中转述 Anthropic 最佳实践及 Claude Code 创建者 Boris Cherny 的说法,称该反馈循环可将结果质量提升 2–3 倍,但该数据来自转述,正文片段在关键实现处被截断。

AI推荐理由 文章聚焦 Claude Code 的一个具体工程问题:模型在测试未通过时仍宣称完成,因为缺少可验证的反馈信号。给出的解法是把测试/构建/linter 等 pass-fail 检查接入工作流,并提到可放在 prompt、CLAUDE.md 等位置。文中引用 Anthropic 最佳实践和 Claude Code 创建者 Boris Cherny 关于反馈循环可提升 2–3 倍结果质量的说法,但这是转述而非本文独立验证,且正文片段在关键处被截断,无法确认 20 行修复的具体实现。对使用 AI 编码 Agent 的开发者有实际参考价值,但证据完整度有限。

62

正文

If you use Claude Code, you've seen this: claude: Done! Partial refunds are implemented. ✅ Enter fullscreen mode Exit fullscreen mode Then you run the tests yourself and four of them fail. Claude isn't lying. It stops when the work looks done, because it has no other signal. If nothing tells it the tests are red, "looks done" is all it has. So you become the test runner: you check, paste the errors back, wait, and check again. This post shows how to make Claude run that check itself, every time, before it's allowed to say it's finished. Why this is the highest-leverage fix Anthropic's best practices for Claude Code put this first: give Claude a way to verify its work, such as tests, a build, a linter or a script, anything that returns pass or fail. Boris Cherny, who created Claude Code, has said that this feedback loop improves the quality of the result 2–3×. There are three places you can put that check: Where How reliable In your prompt ("run the tests after") Works when you remember to type it In CLAUDE.md Advice. Claude usually follows it, sometimes not In a hook Runs every time. Claude can't skip it Hooks are scripts Claude Code runs at fixed points in its loop. The one we want is the Stop hook, which runs whenever Claude tries to end its turn. If a Stop hook exits with code 2, Claude Code doesn't let the turn end. It sends whatever the hook printed to stderr back to Claude, and Claude keeps working. That's the whole trick. The 20-line version Save this as .claude/hooks/verify-gate.py in your project: #!/usr/bin/env python3 import json, os, subprocess, sys data = json.load(sys.stdin) project = os.environ.get("CLAUDE_PROJECT_DIR", os.getcwd()) config = os.path.join(project, ".claude", "verify.txt") if not os.path.exists(config): sys.exit(0) # no check configured: let Claude stop if data.get("consecutive_blocks", 0) >= 4: sys.exit(0) # blocked 4 times in a row: let Claude stop instead of looping command = open(config).read().strip() result = subprocess.run(command, shell=True, cwd=project, capture_output=True, text=True, timeout=300) if result.returncode != 0: tail = (result.stdout + result.stderr).strip().splitlines()[-40:] print("Verification failed:\n" + "\n".join(tail) + "\nFix the failures above, then finish. Do not weaken or skip tests.", file=sys.stderr) sys.exit(2) # exit 2 = block the stop; stderr is sent back to Claude Enter fullscreen mode Exit fullscreen mode Register it in .claude/settings.json: { "hooks": { "Stop": [ { "hooks": [ { "type": "command", "command": "python3 \"${CLAUDE_PROJECT_DIR}/.claude/hooks/verify-gate.py\"", "timeout": 330 } ] } ] } } Enter fullscreen mode Exit fullscreen mode (On Windows, python instead of python3.) Then put your check in .claude/verify.txt: npm test && npx tsc --noEmit Enter fullscreen mode Exit fullscreen mode Restart Claude Code. The next time it says "Done" with red tests, this is what happens: claude: Done. Partial refunds are implemented. Stop hook: Verification failed: ✗ refund.test.ts › partial refund rounds to cents Expected 12.35, received 12.349999 Fix the failures above, then finish. Do not weaken or skip tests. claude: The rounding is off. Fixing src/orders/refund.ts:42… claude: All 48 tests pass and types are clean. Done. Enter fullscreen mode Exit fullscreen mode You didn't paste anything. Claude found out it wasn't done, and fixed it. Writing a good check The hook is only as good as the command in verify.txt. A good one: Exits non-zero on failure. Every normal test runner already does. Runs without prompts. No watch mode, no "press q to quit". vitest needs vitest run, and jest needs --watchAll=false in some setups. Is fast enough to run on every stop. If your full suite takes 10 minutes, point it at the part you're working on: pytest tests/billing -q. Actually covers the feature. Typecheck plus the relevant tests beats either alone. Examples: npm test && npx tsc --noEmit pytest -q && ruff check . go test ./... && go vet ./... cargo test && cargo clippy -- -D warnings Enter fullscreen mode Exit fullscreen mode Windows gotcha: the command runs through cmd.exe, which doesn't treat single quotes as quotes. python -c 'exit(1)' silently passes there, because Python evaluates a string literal and exits 0. Use double quotes inside verify.txt. I found this out when my own README example "passed" a test that should have failed. Where the 20-line version falls short It works, but a real session exposes some edge cases: It runs even when nothing changed. Ask Claude a question, and your whole suite runs before it can answer. Better: skip when git status is clean, or when the tree hasn't changed since the last passing run. A hanging command wedges the turn. timeout=300 raises an exception but can leave child processes running (a dev server, a test watcher). Better: kill the whole process tree, and report the timeout as a failure with advice. consecutive_blocks may not always be there. Keep your own per-session counter as a backup. Crashes should never break a session. Any unexpected error should exit 0 and let Claude stop. I handled all four in a stdlib-only version that works on Windows, macOS and Linux. It's free and MIT licensed: verify-gate-hook on GitHub. Setup is the same three steps. Two habits that make it work better Write the check before the code. If you write verify.txt first and run it once, it should fail, because the feature doesn't exist yet. If it passes, it isn't testing the feature. Tighten it before you start. Watch for weakened tests. The hook's message tells Claude not to weaken or skip tests, and it usually listens. Still skim the diff for edited assertions. A gate is only as honest as the test it runs. Going further The gate answers "is it done?" The harder part is deciding what "done" means before Claude starts. I ended up building a full Claude Code setup around this loop: /lock-in has Claude ask you the hard questions, then write a spec with a single verification command. It writes that command to verify.txt, checks that it fails first, builds step by step against it, and finally has a fresh subagent review the diff against the spec. Hooks block rm -rf ~, force-pushes to main, and reading .env before those commands ever run. /handoff → /clear → /pickup saves your progress to a file and resumes from it in a fresh session, so your context stays small and your usage limit lasts longer. That's the Locked In Kit: 18 skills, 6 subagents, 5 hooks, 8 CLAUDE.md templates and a playbook. The first 20 people get 30% off with code LAUNCH. The hook above is free and works fine on its own, though. Add it to one project today and see how often Claude "finishes" with failing tests. Independent project, not affiliated with Anthropic. Hook behavior is from the Claude Code hooks reference. If something behaves differently in your version, trust the docs.

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容