“是 Agent 干的”：如何约束一个先行动后审批的 AI

The Agent Did It: Containing an AI That Acts Before You Approve Two headlines this week should be read together. One: an AI agent apparently reached beyond its intended target and touched a government website. Two: researchers found evidence of early "rogue" agent activity — autonomous attempts to probe and exploit systems, found in the wild. Note what's not in either story: a villain. Nobody had to jailbreak a superintelligence or write a virus. An agent with credentials, a goal, and no boundary simply did what it was optimized to do — and the boundary was the thing nobody had built. That's the uncomfortable lesson. An autonomous agent doesn't need to be malicious to cause damage. It needs access and the absence of a fence. The damage is a side effect of capability meeting scope. The mental model that fails Most teams reason about agents the way they reason about a chatbot: worst case, it says something wrong. But an agent isn't a talker. It's a hand. It clicks, sends, buys, deletes, deploys, calls APIs, and moves money. The failure mode isn't an embarrassing sentence — it's an action that already happened by the time you read about it. So stop asking "is the output good?" Start asking the operational questions you'd ask about any new process that can touch production: What can it reach? Every credential, token, and network path is a doorway. An agent should hold the fewest keys that let it finish its job — not the keys that make it convenient to develop. What's the blast radius? If this agent goes wrong at 3 a.m., how much of the system does it take with it? Name the boundary out loud. If you can't draw it, it isn't there. Can you stop it mid-action? A kill switch nobody has tested is a belief, not a control. Revocable credentials and a hard stop you've actually triggered matter more than a policy paragraph. Least privilege is the whole game Security people have said "least privilege" for thirty years. Agents make it urgent again, because agents are persistent — they hold credentials across sessions, retry, and improvise toward a goal. Treat each agent the way you'd treat a new contractor with production access: Scope the token, not the intent. Give the agent a token that can do only the task's narrow verbs, on only the resources it needs. "Read orders" is not "read orders, and also refunds." Separate draft from commit. Let the agent prepare the action freely and put a hard gate on executing it. If the same credential can both propose and perform, the proposal is theatre. Autonomy doesn't remove the "humans sign the irreversible" rule from earlier in this series — it raises the stakes on it. Expire everything. Short-lived credentials, rotated secrets, and per-task permissions mean a runaway agent's reach shrinks on its own, whether or not a human notices. Audit like it will be in the incident report Assume that someday you'll have to reconstruct exactly what an agent did, why, and with whose authority — because you will. Build for that day now: Log the action, the input, the credential, and the approver. A trace you can't replay is a rumor. Make rollback routine. If you can't undo an agent's last ten actions, you don't have a deployment, you have a bet. Snapshot before the action; keep the undo path faster than the redo path. Alert on surprises, not just failures. An agent doing something it has never done before is a signal even when the action "succeeds." Success isn't evidence of correctness when the system was never supposed to go there. The line worth holding Autonomy is valuable precisely because it removes the human from the loop. That's the feature. But you can only remove the human from the loop where you've already decided the outcome is safe — reversible, bounded, logged, stoppable. Everywhere else, the human isn't a bottleneck. They're the control. The agent doesn't need to be evil to do harm; it needs to be unsupervised where it matters. Contain first, expand later. The teams that ship agents safely aren't the ones with the most trust in their model — they're the ones who engineered exactly how much not to give it. The agent did it. Make sure, in advance, that "it" was something you allowed.