AI Agent 应在不询问你的情况下被允许做什么？

Your agent notices a failing deployment. It reads the logs, identifies the bad commit, drafts a rollback, and now wants to apply it. Should it just do it? If it asks for permission at every step, it becomes an expensive autocomplete. If it can do anything it can describe, it becomes a liability. The useful question is not “How smart is the agent?” It is: Which actions are safe to pre-authorize, which actions need explicit approval, and which actions should be impossible? That is not a prompt engineering problem. It is a permission design problem. TL;DR An AI agent should be allowed to do low-risk, reversible, private, bounded actions without asking. It should ask before doing anything external, irreversible, expensive, privileged, bulk, legally sensitive, or socially visible. Do not rely on the model to “be careful.” Enforce permissions in your application layer. Use a risk ladder: observe → analyze → propose → draft → execute → publish → destroy. Reads can still be dangerous if they expose secrets, PII, or data the agent can leak. Budgets, rate limits, scopes, and audit logs are part of the permission system. The safest architecture separates agent intent from action authorization. 📋 Table of Contents The real problem: autonomy without authority design 1. “Be Careful” Is Not a Permission Model 2. Use a Risk Ladder, Not a Flat Tool List 3. Reads Are Not Automatically Safe 4. Drafts and Private Writes Are Usually Fine 5. Ask When the Action Crosses a Boundary 6. Budgets and Rate Limits Are Permissions Over Time 7. “Notify Me” Is Not the Same as “Ask Me” 8. Never Let the Agent Grant Itself More Power 9. The Production Pattern: Policy, Audit, and Escape Hatches A practical rule for deciding what to pre-authorize The real problem: autonomy without authority design Most teams start with one of two bad defaults. The first is maximum caution: the agent must ask before everything. That feels safe, but it destroys the value of the agent. If every file read, search query, or draft suggestion requires approval, the human becomes the bottleneck. The second is maximum convenience: the agent gets broad tool access because it is “good at knowing what to do.” That works until the agent misinterprets a request, hits an edge case, follows a poisoned instruction, or performs an action the user never meant. The better approach is to classify actions by risk. An agent can often do these without asking: read non-sensitive logs search internal documentation summarize a file draft a pull request description create a local plan run a read-only query against a scoped replica generate a test in a sandbox An agent should usually ask before doing these: sending an email to a customer publishing a blog post deleting a database record deploying to production refunding a payment inviting a new admin changing IAM permissions contacting an external API that costs money performing a bulk mutation across many records The distinction is not whether the agent is confident. The distinction is whether the action is reversible, bounded, private, and expected. 1. “Be Careful” Is Not a Permission Model Scenario: Your system prompt says, “Only take safe actions. Ask before doing anything destructive.” The agent has a delete_repository tool. One day, it decides a repository is no longer needed. Why it matters: Language model behavior is probabilistic. Prompt instructions influence the agent, but they do not guarantee enforcement. If the tool is available and the agent chooses it, the action may happen. A safe agent system needs an authorization layer outside the model. Solution: Treat every agent action as a permissioned API call. The agent can propose an action, but your system decides whether the action is allowed, requires approval, or is denied. type RiskClass = | "read_public" | "read_sensitive" | "write_draft" | "write_internal" | "external_communication" | "financial" | "destructive" | "privilege_change"; interface AgentActionRequest { agentId: string; userId: string; action: string; risk: RiskClass; resource: string; estimatedCost?: number; context: Record; } interface AuthorizationDecision { allow: boolean; approvalRequired?: boolean; reason?: string; } async function authorizeAgentAction( req: AgentActionRequest ): Promise { const policy = await policyStore.forUserAndAgent(req.userId, req.agentId); if (policy.isDenied(req)) { return { allow: false, reason: "Denied by policy" }; } if (policy.requiresApproval(req)) { return { allow: false, approvalRequired: true }; } if (!(await withinBudget(req))) { return { allow: false, reason: "Budget exceeded" }; } return { allow: true }; } Enter fullscreen mode Exit fullscreen mode Why this works: The model is no longer the final authority. It can suggest delete_repository, but the policy engine can deny it regardless of how persuasive the model’s reasoning is. 🚨 Production warning: If your agent’s safety depends on the system prompt, you do not have safety controls. You have hopeful wording. 2. Use a Risk Ladder, Not a Flat Tool List Scenario: You review agent permissions by looking at a list of tool names: read_logs, create_ticket, send_email, restart_service, delete_branch. Some are obviously dangerous. Others seem harmless. But tool names do not tell you the real risk. Why it matters: A flat tool list hides the important question: what kind of effect does this action have on the world? create_ticket can be harmless if it creates a draft. It can be noisy if it creates a real ticket in a production tracker. It can be serious if it pages on-call at 3 a.m. Solution: Classify actions on a risk ladder. Stage Example Default treatment Observe Read public docs, list files Usually allowed Analyze Summarize logs, compare diffs Usually allowed Propose Suggest a fix, draft a plan Usually allowed Draft Create draft PR, draft ticket, local branch Usually allowed if private Execute internal Update staging config, create internal ticket Allowed with limits Publish Merge PR, publish page, deploy Usually approval Externalize Email customer, post to Slack channel, call partner API Usually approval Spend Buy credits, refund payment, provision paid resources Approval plus budget Destroy Delete record, drop table, terminate instance Strong approval or impossible Change permissions Add admin, modify scopes, change policy Almost always approval This gives you a more stable mental model than tool names. For example: risk_ladder: allow_without_asking: - observe - analyze - propose - draft_private allow_with_limits: - write_staging - create_internal_ticket require_approval: - publish - external_communication - spend - destructive - privilege_change deny: - self_permission_change - policy_modification - bulk_delete_unscoped Enter fullscreen mode Exit fullscreen mode Why this works: You are authorizing effects, not names. A new tool can be added later, but if its effect is “external communication,” it still lands in the approval bucket. 💡 Practical note: When adding a new agent tool, ask: “What is the worst plausible effect if this is called at the wrong time?” Then map that effect to the ladder. 3. Reads Are Not Automatically Safe Scenario: Your agent can read your issue tracker, internal wiki, and customer support database. It is only “reading,” so you allow it. Then a user asks it to summarize a customer issue. The agent includes an API key, internal email, or medical detail in the summary. Why it matters: Read access is still access. If the agent can read sensitive data, it can repeat, transform, summarize, or exfiltrate that data — intentionally or accidentally. This is especially important when the agent can: send messages create public artifacts call external tools write to shared documents generate links or reports operate on untrusted input A read-only action can become a data-leak path. Solution: Apply data classification and field-level filtering before information reaches the agent. interface CustomerRecord { id: string; name: string; email: string; paymentMethod?: string; supportNotes?: string; ssnLast4?: string; } function sanitizeCustomerForAgent( record: CustomerRecord, policy: DataPolicy ): Partial { return { id: record.id, name: policy.canSeePII ? record.name : redact(record.name), email: policy.canSeePII ? record.email : redact(record.email), supportNotes: policy.canSeeSupportNotes ? record.supportNotes : "Support notes unavailable for this agent.", }; } function redact(value: string): string { return `[redacted:${value.length}]`; } Enter fullscreen mode Exit fullscreen mode The important part is that redaction happens before the agent sees the record, not after. Why this works: You reduce the agent’s blast radius. Even if the model is tricked, misused, or overly chatty, it cannot reveal data it never received. ⚠️ Gotcha: “Read-only” is a database property, not a safety property. If the agent can read secrets and then write to a public channel, read access becomes an exfiltration route. 4. Drafts and Private Writes Are Usually Fine Scenario: Your agent wants to create a branch, write a proposed migration, open a draft pull request, or prepare a support response. You are unsure whether these writes should require approval. Why it matters: If every small write requires approval, the agent becomes slow and annoying. But if the agent can publish or execute immediately, mistakes become visible too fast. Solution: Pre-authorize writes that are: private reversible bounded not externally visible not financially consequential not security-sensitive Examples: create a draft branch create a draft document generate a test file in a sandbox write a local plan create an internal note prepare a patch draft a ticket response update a staging-only record A useful pattern is to make the agent’s first write always produce a pending artifact: interface DraftPullRequest { id: string; title: string; body: string; branch: string; status: "draft"; createdBy: "agent"; requiresHumanReview: true; } async function agentCreatesDraftPr(input: { repo: string; branch: string; title: string; body: string; }) { return prService.create({ repo: input.repo, branch: input.branch, title: input.title, body: input.body, draft: true, metadata: { createdBy: "agent", requiresHumanReview: true, }, }); } Enter fullscreen mode Exit fullscreen mode Why this works: The agent can make progress without forcing a human to approve every microstep. The human reviews the artifact before it becomes real. This is one of the best autonomy patterns: let the agent do the work, but make the result inherently reviewable. 🧠 The important part: Drafts are powerful because they separate labor from commitment. The agent can do the labor. A human or policy can control the commitment. 5. Ask When the Action Crosses a Boundary Scenario: The agent is asked to “clean up old customer accounts.” It finds 3,000 inactive accounts and prepares a deletion batch. The user meant “mark inactive,” not “delete.” Why it matters: Some actions are not dangerous because they are technically complex. They are dangerous because they cross a boundary: from private to public, from reversible to irreversible, from cheap to expensive, from internal to external. The approval question is really a boundary question. Solution: Require approval when an action crosses one of these boundaries: External visibility email customer post to Slack channel publish article create public comment send webhook to partner Irreversibility delete record terminate instance revoke token cancel subscription merge without rollback plan Financial impact refund payment buy credits provision paid resources upgrade plan trigger billable API Privilege escalation add admin change OAuth scopes modify IAM policy install app with broad permissions Bulk impact update all users delete stale records migrate production table send campaign to 50,000 recipients A policy gate can look like this: function requiresApproval(req: AgentActionRequest): boolean { if (req.risk === "external_communication") return true; if (req.risk === "financial") return true; if (req.risk === "destructive") return true; if (req.risk === "privilege_change") return true; if (req.context.bulk === true) return true; if (req.context.environment === "production" && req.action.includes("delete")) { return true; } return false; } Enter fullscreen mode Exit fullscreen mode Why this works: You are not asking the model to judge every situation. You are encoding boundaries that your organization already understands. 🔍 Why this matters: Approval should be triggered by the nature of the action, not by the model’s confidence. Confidence is not authority. 6. Budgets and Rate Limits Are Permissions Over Time Scenario: Your agent is allowed to call a search API, run code in a sandbox, and retry failed steps. Individually, each action seems safe. But a bug causes it to retry in a loop. Within an hour, it has burned through API credits and created hundreds of duplicate artifacts. Why it matters: A single action can be safe while the cumulative behavior is unsafe. Permissions need to account for frequency, volume, and cost. An agent that can spend $0.05 per action is not safe if it can perform 100,000 actions. Solution: Attach budgets and rate limits to agent capabilities. interface BudgetState { agentId: string; day: string; estimatedCostUsd: number; toolCalls: number; } async function withinBudget(req: AgentActionRequest): Promise { const budget = await budgetStore.get(req.agentId); const estimatedCost = req.estimatedCost ?? 0; if (budget.toolCalls + 1 > policy.maxToolCallsPerDay) { return false; } if (budget.estimatedCostUsd + estimatedCost > policy.maxDailyCostUsd) { return false; } return true; } Enter fullscreen mode Exit fullscreen mode For rate limits: async function withinRateLimit(req: AgentActionRequest): Promise { const key = `rate:${req.agentId}:${req.action}`; const count = await rateLimiter.increment(key, { window: "1m" }); return count <= policy.maxCallsPerMinute(req.action); } Enter fullscreen mode Exit fullscreen mode Why this works: You are treating autonomy as a bounded resource. The agent can act without asking, but only within a container of acceptable consumption. This is especially important for agents that can: call paid model APIs run compute-heavy jobs provision cloud resources send messages scrape external sites retry automatically spawn sub-agents 💡 Practical note: If an agent can retry, it needs a circuit breaker. Unlimited retries turn small mistakes into expensive incidents. 7. “Notify Me” Is Not the Same as “Ask Me” Scenario: Your agent sends a message saying, “I deleted the stale staging environment.” The user sees the notification two hours later. The environment contained a database seed needed for a demo. Why it matters: There is a big difference between: blocking until the user approves notifying the user after the action logging the action silently Many systems blur these together. They say the agent “asked” when it really just announced. Solution: Be explicit about the approval mode. type ApprovalMode = | "allow" | "notify_after" | "require_approval_before" | "deny"; interface ApprovalRequest { runId: string; agentId: string; userId: string; action: AgentActionRequest; mode: ApprovalMode; expiresAt: string; } async function handleAction(req: AgentActionRequest, runId: string) { const decision = await authorizeAgentAction(req); if (decision.allow) { return executeAction(req); } if (decision.approvalRequired) { const approval = await approvalStore.create({ runId, agentId: req.agentId, userId: req.userId, action: req, mode: "require_approval_before", expiresAt: new Date(Date.now() + 30 * 60_000).toISOString(), }); await runStore.pause(runId, approval.id); return { status: "waiting_for_approval", approvalId: approval.id, }; } throw new Error(decision.reason ?? "Action denied"); } Enter fullscreen mode Exit fullscreen mode Now the run can pause until approval arrives. Why this works: The system distinguishes between actions that can proceed, actions that need a human gate, and actions that merely deserve a record. Useful defaults: Action type Best mode Read public docs Allow Summarize internal data Allow or notify Create draft PR Allow Create real ticket Notify or approval depending on impact Send customer email Require approval before Delete production data Require approval before or deny Change permissions Require approval before Spend money Require approval before ⚠️ Gotcha: If the action cannot be undone, notifying afterward is not the same as consent. 8. Never Let the Agent Grant Itself More Power Scenario: Your agent can manage tools, install plugins, update its own configuration, or request new OAuth scopes. It encounters a task it cannot complete, so it tries to expand its permissions. Why it matters: This is the permission equivalent of giving a program sudo because it asked nicely. If an agent can modify its own authority, the entire permission model collapses. This risk shows up in several forms: the agent installs a new MCP server or plugin the agent requests broader API scopes the agent updates its own policy file the agent creates a new admin user the agent writes a workflow that runs with higher privileges the agent asks another agent with more permissions to perform the action Solution: Make permission changes a protected human action by default. const PROTECTED_ACTIONS = new Set([ "policy.update", "agent.scope.grant", "oauth.scope.request", "user.role.promote", "tool.install", "mcp_server.register", "workflow.privilege.escalate", ]); function isProtectedAction(action: string): boolean { return PROTECTED_ACTIONS.has(action); } async function authorizeProtectedAction(req: AgentActionRequest) { if (isProtectedAction(req.action)) { if (req.context.requestedBy === req.agentId) { return { allow: false, reason: "Agents cannot modify their own privileges", }; } return { allow: false, approvalRequired: true, reason: "Privilege changes require human approval", }; } return authorizeAgentAction(req); } Enter fullscreen mode Exit fullscreen mode Why this works: You prevent the agent from becoming a privilege-escalation vector. Even if the agent is compromised, confused, or manipulated by untrusted input, it cannot directly widen its own authority. 🚨 Production warning: Watch out for indirect escalation. If Agent A can ask Agent B to do something, and Agent B has broader permissions, you need to evaluate the full delegation chain. 9. The Production Pattern: Policy, Audit, and Escape Hatches The safest agent systems I’d design do not try to make the agent perfectly wise. They make the system legible and controllable. A practical production architecture looks like this: User request ↓ Agent planner ↓ Proposed action ↓ Policy engine ↓ Budget/rate check ↓ Approval gate if needed ↓ Tool execution layer ↓ Audit log ↓ User notification / UI Enter fullscreen mode Exit fullscreen mode The important components are: Policy engine Decides whether an action is allowed, denied, or needs approval. default: deny rules: - allow: action: logs.read environment: staging - allow: action: pull_request.create_draft repo_visibility: internal - require_approval: action: email.send recipient_type: external - require_approval: action: payment.refund when: amount_cents_gt: 10000 - deny: action: policy.update requested_by: agent Enter fullscreen mode Exit fullscreen mode Execution layer Performs the actual tool call only after authorization. Audit log Records: who requested the action which agent proposed it what policy applied what approval was given what arguments were used what result occurred what cost was incurred Escape hatch Lets a human pause, cancel, or roll back the agent. await runStore.cancel(runId, { reason: "user_requested", cancelledBy: userId, }); Enter fullscreen mode Exit fullscreen mode Why this works: You are not asking the model to be the security boundary. You are building a system where the model can be useful without being omnipotent. A practical rule for deciding what to pre-authorize If I had to reduce the decision to one rule, it would be this: An AI agent may act without asking when the action is low-risk, reversible, bounded, private, auditable, and within a pre-approved budget. If any of those are false, pause. More concretely, I would allow an agent to act without asking when all of these are true: The action does not affect external users. The action does not spend significant money. The action does not delete or overwrite important data. The action does not change permissions. The action is scoped to a single resource or small batch. The action can be reviewed or undone. The action does not expose sensitive data. The action is within rate and cost limits. The user would reasonably expect the agent to do it. I would require approval when any of these are true: The action sends something to a human. The action publishes something publicly. The action affects production. The action costs meaningful money. The action is hard or impossible to reverse. The action changes access control. The action touches regulated or highly sensitive data. The action affects many records at once. The action was derived from untrusted content. The last one deserves emphasis. If the agent read an email, issue comment, web page, or document that could contain hostile instructions, you should be more cautious about letting it take privileged actions immediately afterward. The goal is not to make agents passive. The goal is to make their autonomy legible. A good agent permission system feels like a well-designed employee role: enough access to be useful not enough access to cause a disaster clear escalation paths a complete record of what happened The agent should not be trusted because it sounds confident. It should be allowed because your system has already decided that this class of action is safe.