你的 Agent 不需要写一篇小作文来做决策

Something interesting appeared in a recent LocalLLaMA project showcase. Several projects were exploring small models that do not generate prose. They take a state, a question, and a fixed set of choices, then return a probability for each choice. That sounds less impressive than an autonomous agent narrating its plan. It may also be exactly what many agent loops need. Consider the decisions inside a typical tool-using workflow: Which tool should handle this request? Is this command safe enough to run automatically? Did the previous step succeed? Should the workflow retry, stop, or ask a person? Which retrieved document should be examined next? These are closed-set decisions. Asking a generative model to produce a paragraph, parsing that paragraph, and hoping it stayed inside the allowed choices is a surprisingly elaborate way to select one value from an enum. My current view is simple: use generative models for open-ended reasoning and content. Use constrained decision components when the answer space is already known. Generation and decision are different jobs A generative model is useful when the output cannot be enumerated in advance. Writing a migration plan, diagnosing an unfamiliar failure, explaining a trade-off, or proposing a patch all need flexible output. Routing a ticket to one of twelve queues does not. We often use the same large model for both jobs because the API is convenient. The model receives a prompt such as: Choose one action: retry, stop, escalate. Return JSON only. Enter fullscreen mode Exit fullscreen mode The application then validates the response, repairs malformed JSON, rejects invented actions, and perhaps asks the same model to try again. We added a language generator, then built a fence around its language generation. Structured output improves the interface, but it does not change the underlying job. The system still generates tokens to choose from a fixed set. Projects such as Contrastive Language Models approach the problem differently. CLM encodes the current state and candidate actions, scores their relationship, and returns a distribution over the supplied actions. The repository describes uses such as tool routing, trajectory ranking, and retrieval shortlisting. Decider takes another route. It reads a state and typed questions, then produces probabilities for fixed choices, yes-or-no decisions, or described score levels in one pass. Its core promise is deliberately narrow: no free-form decoding and no output outside the choices supplied by the application. These projects are early and their benchmark claims need independent validation. The architecture is still worth paying attention to. The application should own the choice set In a production agent, the model should not decide which actions exist. The application should. application state + allowed actions ↓ decision component ↓ probabilities ↓ policy and threshold ↓ execute, ask, or stop Enter fullscreen mode Exit fullscreen mode That boundary gives us several useful properties. First, impossible actions stay impossible. If delete_customer is not in the candidate set, the decision component cannot select it. This is stronger than telling a model in a prompt not to mention deletion. Second, the result is inspectable. A distribution such as this is easier to evaluate than a persuasive paragraph: { "retry": 0.18, "stop": 0.07, "escalate": 0.75 } Enter fullscreen mode Exit fullscreen mode Third, the application can apply policy after inference. A high score is evidence, not authority. if decision == "retry" and probability >= 0.90 and retry_count = 2 does not need inference. A model becomes useful when the state contains language or evidence that code cannot classify reliably with a small, stable rule set. Run shadow evaluations before granting automation. Record the model's decision and probability without executing it. Compare that output with the actual operator or workflow decision. This shows where a threshold would automate safely and where it would merely automate confidence. Do not create a second platform by accident There is a predictable failure mode here. A team sees specialized decision models, creates a separate serving stack, a feature store, a training pipeline, an evaluation service, and a new control plane before proving that one decision benefits from any of it. Start with one bounded choice that occurs frequently enough to matter. Tool routing is a reasonable candidate. Define the allowed tools, create a representative test set, compare rules and models, and measure error cost. Serve a smaller model only if it produces a material latency, cost, privacy, or accuracy benefit. For low-volume workflows, a general model with strict structured output may remain the simplest operational choice. Fewer services can be worth a little extra inference cost. The recommendation changes when decision calls dominate the loop. If an agent makes dozens of small choices per task, generation latency and token cost accumulate. A fast decision component can then reduce both, while giving the application a consistent probability interface. Keep the agent for the parts that need an agent Open-ended work still benefits from capable generative models. A decision model will not investigate an unfamiliar production failure, compare architecture options, or write a useful migration plan. It can decide which diagnostic to run next from an approved list. It can estimate whether the evidence supports another step. It can route uncertainty to a person before the workflow turns a weak guess into an action. That is enough. An agent does not become less intelligent because some decisions move into smaller, constrained components. The system becomes clearer about where intelligence is needed, where policy belongs, and where ordinary code is still the better tool. If the answer must be one of five choices, make the system choose one of five choices. Save the essay for the part that needs an explanation.