智能体
dev.to ·
@Adam McClarin
从未运行的绿色测试套件不构成证据
作者在指导 AI 编码 Agent 处理多周重构、类型化契约与影子路径时发现,最危险的问题不是崩溃而是静默失效:溯源日志数月零字节、测试未执行却显示绿色、否决逻辑持续丢弃有效配置。作者主张把 Agent 的报告视为待验证声明,每个否定结果都需阳性对照,先做变异测试确认测试能捕获真实故障。
AI推荐理由 作者基于指导 AI 编码 Agent 的实战经验,指出测试套件“绿色”但未真正执行、溯源日志长期零字节等静默失效问题,提出“每个否定结果需阳性对照、先做变异测试”的实践方法,对使用 AI 编码 Agent 的开发者有具体借鉴意义;属个人经验分享,非系统性研究。
60
正文
I have spent a stretch of weeks on a live system where the dangerous part was not a crash.
It was quiet. A provenance log that had written zero bytes for months. Tests that scored green without executing. A veto that had been dropping valid setups while the board still looked calm. A calendar built for the wrong exchange.
None of that announces itself. The UI still loads. The agent still reports progress. Someone still says the suite is green.
I direct AI coding agents on work like this. Multi-week rewires. Typed contracts. Shadow paths. The agent is fast. The agent will also tell you a push landed when you have not checked the remote, or that a cost number is solid when it was inferred from a fixture. I treat the report as a claim. Then I check the claim.
The practice I now keep is small.
Every negative result needs a positive control. If the test says nothing happened, I need proof the test can detect a real break. Mutation first. Break the thing the fix is supposed to catch. Confirm the suite sees it. Restore.
Distinguish measurement from inference. An extraction bug is not a market finding. A time zone error is not a data gap. Write the correction down before it becomes folklore.
On a live rewire, keep the old path authoritative until the new path is proven in shadow. One boundary at a time. Do not cut over on confidence.
If you are using an agent on a system that already carries money, records, or a book, the skill is not the prompt. The skill is knowing what to re-run yourself.
I do this work at Meraki is Love. Studio site: merakislove.com