AI Agent 应该何时停止？

Designing a progress-aware execution controller for agentic systems. Imagine asking an AI research agent to compare PostgreSQL and DynamoDB. It finds documentation, collects the important trade-offs, and has enough information to answer. Then it searches again. And again. Same sources, slightly different queries, more tokens spent. Now consider the opposite: an agent reaches its step limit one action before completing its work. Both situations point to the same problem: executing more steps isn't always the same as making progress. I've been thinking about how I'd design an execution controller that makes stopping a system-level decision, not simply another instruction in the prompt. Why max_steps = 10 doesn't solve it Hard limits are essential. They stop runaway loops and cap resource consumption. But a fixed number of turns doesn't tell us whether a task is complete. A simple lookup might need two turns; a difficult research task might need twenty. A generous limit wastes resources on easy tasks. A tight limit can cut complex tasks short. The limit should be the safety boundary, not the intelligence behind the stopping decision. The architecture I'd try I'd separate the agent from the component that controls its execution. The agent runtime proposes actions. A policy and budget gate checks them before execution. After a tool returns, a progress evaluator measures what changed, and a decision controller decides what happens next. The controller tracks three practical signals: Completion: Which task requirements have actually been satisfied, with evidence? Progress: Did the last action add useful information or just repeat earlier work? Budget: How much time, money, and tool usage is left? For example, three searches returning the same sources with no newly satisfied requirements could be a sign of stagnation. But it shouldn't immediately kill the task; sometimes a different strategy is all that's needed. Stop, continue, or change course? I would start with a transparent, rule-based policy rather than making another LLM judge every step. The outcomes are deliberately different: FINISH - all mandatory requirements are verified. CONTINUE - useful progress is still being made. REPLAN - the agent appears stuck, but another strategy is allowed. STOP INCOMPLETE - the budget is exhausted or continuing is unsafe. I think that final distinction matters. An agent stopping because it ran out of budget should never be confused with an agent successfully finishing its job. One detail that matters in production The controller's state should be durable: completed requirements, evidence references, tool history, and consumed budget. If a worker crashes after spending $0.30 of a $0.50 budget, restarting shouldn't silently reset that budget. And if the previous action changed external state, retries need idempotency or reconciliation-not merely a checkpoint. For an MVP, I'd use a small agent runtime, PostgreSQL for execution records, and deterministic policy checks. No elaborate distributed architecture unless the workload actually needs it. How I'd know if it works I wouldn't assume a smarter-looking controller is better. I'd test three policies on the same set of tasks: Policy Behavior Fixed limit Stop on completion or at a hard cap Stagnation-aware Replan after repeated non-progress Progress-aware Combine completion evidence, stagnation, and soft budgets I'd compare verified task success, cost per successful task, latency, and premature stops. The useful question isn't whether the controller reduces calls. It's whether it saves wasted work without making the agent less reliable. A simple limit might even win on predictable tasks. That's a valid engineering result too. Final thought I think the next step for agentic systems isn't just making agents capable of doing more. It's giving the systems around them a better way to decide when more work is useful, when to change strategy, and when to stop. This is a proposed architecture, not a benchmarked result. I'd like to turn it into a small experiment and see where the simple rules break. Sometimes better agent behavior means doing less-but doing it deliberately.