测试输出淹没 Agent 上下文：9 个参数裁剪其读取内容

Book: LLM Observability Pocket Guide Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Your agent runs the test suite. One test fails. Hundreds pass, and the runner tells the agent about every one of them: a header, progress for each file, the names of passing tests, a summary, and somewhere in there the stack trace it actually needs. Test reporters were written for a person scrolling a terminal. A coding agent doesn't scroll. It reads what the command printed. The useful part can get cut. A command that fails reaches Claude Code inline only up to roughly 10,000 characters. Past that, Claude gets "a head-and-tail excerpt of that size", with no file path to the rest. A red test run exits non-zero, so it counts as a failure, and it is the run where the middle matters most. And whatever fits can stay. Claude Code "sends your full conversation with every request", so the names of passing tests can ride along on later turns until the conversation is compacted or cleared. You can't change how the agent reads. You can change what the runner prints. Measure what your agent reads first Run the suite the way an agent does: piped, not in an interactive terminal, with stderr merged into stdout. Then count. npx vitest run 2>&1 | wc -c npx vitest run --reporter=minimal 2>&1 | wc -c Enter fullscreen mode Exit fullscreen mode wc -c counts bytes. For plain ASCII output that is also characters, the unit Claude Code's limits use. I'm not printing numbers here, because they depend entirely on your suite: how many tests, how much they log, how deep the stack traces go. Run it on your project, before and after each flag below, and keep the flags that move your count. A small loop compares several variants in one go: for cmd in "pytest" "pytest -q" "pytest -q --tb=short"; do printf '%-24s' "$cmd" sh -c "$cmd" 2>&1 | wc -c done Enter fullscreen mode Exit fullscreen mode sh -c runs each string as a full command, flags included, in bash or zsh. Run this from your own shell, not inside an agent session. Vitest changes its reporter by itself when it detects an AI coding agent (flag 1), so measuring inside the agent hides the "before". JavaScript and Python: flags 1 to 4 1. Vitest: --reporter=minimal Vitest 4.1 added a reporter "to reduce ai agent token usage". The current reporters guide calls it minimal, with agent as an alias. It prints only failed tests and their error messages, and suppresses console logs from passing tests and the summary section. When no reporter is configured and Vitest detects an AI coding agent, it uses this one automatically. Pass it yourself when your config sets reporters, or when detection doesn't recognise your agent: npx vitest run --reporter=minimal Enter fullscreen mode Exit fullscreen mode If you want a progress signal, --reporter=dot prints "a single dot for each completed test", with details only for failures plus the summary. And the CLI's --silent=passed-only keeps console output from failing tests only. 2. Jest: --silent If your code logs while tests run, every one of those lines lands in the agent's context. Jest's CLI has one flag for that: --silent will "Prevent tests from printing messages through the console." npx jest --silent Enter fullscreen mode Exit fullscreen mode Jest has a default that bites agents: verbose is false, "or true if there is only one test file to run". The quick rerun of one file after a fix, the run agents do all the time, is the run that reports each test individually. 3. pytest: -q -q is pytest's "quiet - less verbose - mode". The flag to look for in your scripts and addopts is the opposite one: -v gives each test its own line, and -vv stops truncating assertion text entirely. 4. pytest: --tb=short The default, --tb=auto, prints "'long' tracebacks for the first and last entry". --tb=short is the "shorter traceback format", and --tb=line is "only one line per failure". Short keeps the file, the line and the failing assertion, which is usually what the agent needs to act. Line is often too little. pytest -q --tb=short Enter fullscreen mode Exit fullscreen mode One more from the same page: if addopts turns on --showlocals, every traceback carries the local variables. --no-showlocals hides them for that run. Go, Rust, PHP and Ruby: flags 5 to 8 5. Go: take -v off go test ./... is already quiet: a passing package prints only the final ok summary line. The -v flag logs all tests as they run, and will "Also print all text from Log and Logf calls even if the test succeeds." If a Makefile target or the agent adds -v, every passing test and every t.Log comes back. If you want a friendlier view, gotestsum wraps go test -json. Its default pkgname format prints one line per package, and dots prints one character per test. Flags for go test go after --: go test ./... gotestsum --format dots -- ./... Enter fullscreen mode Exit fullscreen mode pkgname grows with packages and dots grows with tests, so measure both on your repo. 6. Cargo: -q, on both sides of -- cargo test -q -- --quiet Enter fullscreen mode Exit fullscreen mode The first -q is Cargo's own and hides its log messages. Everything after -- goes to the test harness. There, libtest's --quiet "Displays one character per test instead of one line per test" and is an alias for --format=terse. The default, pretty, is one line per test. 7. PHPUnit: --no-progress PHPUnit's TextUI docs list --no-progress: "Disable output of test execution progress." Failures and the result summary still print. vendor/bin/phpunit --no-progress Enter fullscreen mode Exit fullscreen mode Two neighbours to avoid in an agent loop: --testdox will "Replace default result output with TestDox format", which reports each test by name, and --no-output will "Disable all output", which takes the failure details with it. 8. RSpec: --format progress RSpec defaults to the progress formatter: . for a pass, F for a failure, * for pending. The documentation formatter prints the text of every describe and it instead. Check the project's .rspec file: if it has --format documentation, change it to --format progress or delete the line. Flag 9: stop at the first failure Every runner above has a version of the same switch. When the agent is fixing one thing, the failures after the first are output it reads before it can act on any of them. Runner Stop early What the docs say Vitest --bail=1 stop after N failed tests Jest --bail N failing suites, default 1 pytest -x stop after first failure go test -failfast no new tests after one fails PHPUnit --stop-on-failure stop on first failure RSpec --fail-fast stop on first failed test Sources: Vitest CLI, Jest CLI, pytest, go test, PHPUnit, RSpec. Cargo stops early at a coarser level: it exits after the first test executable fails, but libtest runs every test inside that executable. Libtest's own --fail-fast exists, and it needs -Z unstable-options. Put together, the fast loop looks like this: npx vitest run --reporter=minimal --bail=1 npx jest --silent --bail pytest -q --tb=short -x go test -failfast ./... cargo test -q -- --quiet vendor/bin/phpunit --no-progress --stop-on-failure bundle exec rspec --format progress --fail-fast Enter fullscreen mode Exit fullscreen mode Stopping early is for the loop, not the finish line. Before the agent says it's done, it should run the whole suite, quietly, and read every failure. Pin the commands, then watch the meter Flags help only if the agent uses them. Put the commands in the file your agent reads when a session starts. Claude Code loads CLAUDE.md "into context at session start". AGENTS.md is the open format that Codex, Gemini CLI, Cursor, Aider and others read. ## Tests - While fixing: `pytest -q --tb=short -x tests/test_orders.py` - Before saying done: `pytest -q --tb=short` Enter fullscreen mode Exit fullscreen mode Claude Code's cost guide lists two heavier options too: a PreToolUse hook that rewrites test commands so only failures come back, and running tests in a subagent so "the verbose output stays in the subagent's context while only a summary returns". Then you want to see the effect. I build Hermes IDE, so weigh this part with that in mind. Hermes runs Claude Code, Codex and other CLI agents in real terminals, and version 2.3.4 added a context meter that shows how full the conversation is and offers Compact when it gets close to full. Hermes 2.3.4: Claude Code's /context breakdown after pressing the Context chip, per-session context gauges in the sidebar, and the token total and estimated cost in the status strip. Run the suite once with default flags and once with the quiet ones, and watch the meter between turns. Compact is there when you need it, in the Commands list above each session. The action bar above a Claude Code session, with the Commands list open: pinned Context, Model, Plan and Review, then built-in commands such as Compact, Usage and Diff. Compacting is the fix after the fact, and not a free one: Claude Code's docs note that /compact "reads the conversation it summarizes, so compacting a large context is itself a large request." The flags above are the cheaper fix. Hermes is free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. Download it here, and the code is on GitHub. If this was useful Counting bytes with wc -c is the crude version of a question that gets harder as a team grows: what did the model actually read, and what did that cost? The LLM Observability Pocket Guide compares the tracing and eval tools that answer it across a whole system, not one terminal.