最新资讯
bloomberg.com Nvidia 加强东南亚芯片追踪以打击走私 thestack.technology 模型路由需求升温,Agent Router 推出开源替代方案 unite.ai Cashfree 为 AI Agent 的托管结账加入 WebMCP 工具 dev.to 我们在 DolphinBench 上测试记忆检索:纯关键词搜索暴露了自身回归 dev.to 别再把 Agent 写成 while 循环,把它建模为状态机 analyticsindiamag.com 小语言模型可能重塑企业 AI 部署方式 bloomberg.com Rightmove 与 Scout24 借 AI 产品应对低迷房市并上调展望 stackoverflow.blog 欢迎来到芯片时代:半导体制造、AI 数据中心与芯片设计的未来 www.youtube.com AI Agent 总犯同样的错:CentralAgent AI 将记忆重构为经验系统 pub.towardsai.net Netflix 科学家建议重采样 LLM 评判:5 次仅将误差棒缩小 13% pub.towardsai.net Jev 作为 LLM 前的静默层:构建提示感知的模型路由器 reddit.com OpenAI 称 Astra 仅满足 10% 安全任务标准
多元动态
登录 注册
智能体 dev.to · @Peeyush Kant Misra

你的 AI Agent 现在能调用 API 了,但谁来审查它的行为?

原文 我的收藏

文章指出,agentic AI 的真正问题不在于给 LLM 提供工具,而在于授予访问权限后如何控制其行为。当 Agent 可访问 GitHub、数据库、Slack、Jira、AWS、邮件和内部 API 并执行读取数据、调用接口、修改代码、提交 PR、发送消息等操作时,谁来决定它实际被允许做什么成为关键工程问题。作者认为 MCP、授权机制、运行时验证和 Agent 安全正成为 agentic AI 时代最重要的工程挑战之一。

AI推荐理由 文章聚焦 agentic AI 的权限与运行时验证问题,提出 MCP、授权、运行时校验等具体工程议题,属于开发者视角的观点性讨论;但正文仅为开头片段,未给出具体方案、案例或可验证证据,观点深度有限。

62

正文

Your AI Agent Can Call APIs Now. Who Is Checking What It Does? The real problem with agentic AI isn't giving an LLM tools. It's controlling what happens after you give it access. Imagine you give an AI agent access to: your GitHub your database Slack Jira AWS email internal company APIs And then you tell it: "Fix the production issue." The agent doesn't just generate text anymore. It can *read data, call APIs, create files, modify code, open pull requests, send messages, and potentially trigger real-world actions. * That's incredibly powerful. But it creates a new question that we don't talk about enough: When an AI agent has permission to act, who decides what it is actually allowed to do? This is where MCP, authorization, runtime verification, and agent security become extremely important. And this is quickly becoming one of the biggest engineering problems in the agentic-AI era. From Chatbots to Agents A traditional chatbot mostly follows this pattern: User ↓ LLM ↓ Response Enter fullscreen mode Exit fullscreen mode You ask: "Explain Kubernetes." The model generates an answer. An AI agent is different. It can look more like this: > User > ↓ > AI Agent > ↓ > Reasoning > ↓ > Choose Tool > ↓ > Call API > ↓ > Observe Result > ↓ > Reason Again > ↓ > Call Another Tool > ↓ > Take Action Enter fullscreen mode Exit fullscreen mode The important change is the final step. The model isn't just telling you what to do. It can potentially do it for you. That's the fundamental shift behind agentic AI. The Tool Is Where Things Get Interesting Suppose we give our agent three tools: search_database() send_email() create_github_pr() Enter fullscreen mode Exit fullscreen mode The LLM might decide: User request ↓ "Investigate failed deployment" ↓ search_database() ↓ Find error ↓ create_github_pr() ↓ Fix proposed ↓ send_email() ↓ Notify developer Enter fullscreen mode Exit fullscreen mode From the user's perspective: "The AI fixed my problem." From the infrastructure's perspective: An AI-controlled identity just accessed multiple systems and performed multiple actions. That's a very different security problem. Enter MCP This is one reason Model Context Protocol (MCP) has become so important. MCP provides a standardized way for AI applications to interact with external capabilities such as tools and resources. Instead of building a completely custom integration for every AI application and every service, MCP gives developers a common protocol for exposing capabilities to AI systems. The protocol has evolved significantly in 2026. The July 28, 2026 specification introduced a stateless protocol core, improved authorization mechanisms, cacheable discovery results, Tasks, and an extensions framework designed for a more production-oriented ecosystem. But here's the important distinction: MCP can standardize how an agent interacts with tools. It does not magically make those interactions safe. And that's where things get complicated. Giving an Agent a Tool Is Giving It a Capability Consider this MCP tool: { "name": "delete_user", "description": "Delete a user account", "inputSchema": { "type": "object", "properties": { "user_id": { "type": "string" } } } } Enter fullscreen mode Exit fullscreen mode Technically, your agent can now call: delete_user("user_123") Enter fullscreen mode Exit fullscreen mode But should it? That's a different question. The LLM might have permission to discover the tool. That doesn't necessarily mean it should have permission to execute the tool. And even if it can execute it, maybe the operation should require: User confirmation ↓ Policy check ↓ Authorization ↓ Tool execution ↓ Audit log Enter fullscreen mode Exit fullscreen mode That's the layer we're going to need more of. The Problem With "The LLM Will Decide" One dangerous architecture looks like this: User ↓ LLM ↓ "Seems safe" ↓ Tool ↓ Production System Enter fullscreen mode Exit fullscreen mode We're essentially asking the same system that decides what action to take to also decide *whether that action should be allowed. * That's not a great security boundary. A better architecture separates these responsibilities. ┌───────────────┐ │ User │ └───────┬───────┘ ↓ ┌───────────────┐ │ AI Agent │ └───────┬───────┘ ↓ ┌───────────────┐ │ Policy Layer │ └───────┬───────┘ ↓ ┌─────────┴─────────┐ ↓ ↓ Allowed? Blocked? ↓ ↓ Execute Stop ↓ Tool / API ↓ Audit Log Enter fullscreen mode Exit fullscreen mode The key idea: The LLM proposes. A deterministic control layer decides. Why This Matters With MCP Current DEV discussions are already moving beyond simply "How do I build an MCP server?" toward questions such as: How should MCP tool calls be verified? How should agent permissions be enforced? What happens when an MCP server is malicious? How do we prevent tool abuse? How do we audit autonomous actions? DEV currently has an active trend around AI-agent security and authorization, including posts on MCP tool-call policy gates and malicious MCP server supply-chain risks. That's a significant shift. The conversation is moving from: "Can agents use tools?" to: "Can we safely let agents use tools?" Think About an AI Agent Like a New Employee Here's an analogy I really like. Imagine hiring a new developer. You wouldn't give them: AWS admin + Production database + GitHub organization owner + Company email + Payroll system Enter fullscreen mode Exit fullscreen mode on their first day. You'd give them limited permissions. Maybe: GitHub → Read + PR creation Database → Read-only AWS → Development account Production → No access Email → No access Enter fullscreen mode Exit fullscreen mode And you'd monitor their actions. AI agents should be treated similarly. Instead of: Agent = Administrator we should think: Agent = Identity + Capabilities + Policies + Audit Least Privilege Becomes Even More Important Traditional security already follows a principle called: Least privilege Give a user or service only the permissions required to perform its job. For agents, this becomes even more important because an agent can potentially chain multiple tools. For example: Tool A → Read customer data Tool B → Create document Tool C → Send email Individually, these capabilities might appear harmless. But combined: Read sensitive customer data ↓ Put it into document ↓ Email document externally Enter fullscreen mode Exit fullscreen mode The dangerous behavior may not exist inside any single tool. It can emerge from tool composition. That's one of the biggest security challenges in agentic systems. And Then There's the MCP Supply Chain Here's another problem. Developers can install or connect MCP servers created by third parties. That means we're introducing something similar to a dependency ecosystem. Think about npm packages. You don't blindly trust every package because it has a nice README. The same principle should apply to agent tools. A malicious or compromised MCP server could potentially become part of an agent's execution environment. That is why current developer discussions are increasingly focused on MCP supply-chain security and trust. The New Security Stack I think production agent architectures will increasingly look something like this: ┌──────────────────┐ │ User │ └────────┬─────────┘ ↓ ┌──────────────────┐ │ AI Agent │ └────────┬─────────┘ ↓ ┌──────────────────┐ │ Policy / Guard │ │ Layer │ └────────┬─────────┘ ↓ ┌──────────────────┐ │ Authorization │ └────────┬─────────┘ ↓ ┌──────────────────┐ │ MCP │ └────────┬─────────┘ ↓ ┌────────────┼────────────┐ ↓ ↓ ↓ GitHub Database AWS │ │ │ └────────────┼────────────┘ ↓ Audit / Logs Enter fullscreen mode Exit fullscreen mode This is much closer to how we should think about production agent infrastructure. What Should Developers Actually Do? If you're building an AI agent today, don't stop at: agent = create_agent() agent.add_tool(...) agent.run(...) Enter fullscreen mode Exit fullscreen mode Ask these questions instead. 1. What can the agent access? List every tool. GitHub Database Slack AWS Email Filesystem Browser What can each tool actually do? Don't just ask: "Does the agent have access to GitHub?" Ask: "Can it read repositories?" "Can it create branches?" "Can it merge PRs?" "Can it delete repositories?" Those are very different capabilities. 3. Which actions require confirmation? For example: Read database → automatic Create draft PR → automatic Merge PR → confirmation Delete production DB → impossible Send external email → confirmation Enter fullscreen mode Exit fullscreen mode 4. Can you audit every action? You should be able to answer: Which agent? Which user? Which tool? Which arguments? Which resource? When? Why? What happened? If you can't answer these questions, debugging an autonomous system becomes extremely difficult. The Interesting Future MCP is evolving beyond a simple way of connecting an LLM to a tool. The 2026 specification is explicitly addressing things such as authorization, scalability, long-running tasks, extensions, and lifecycle management. And the ecosystem is exploring mechanisms around stronger trust and auditability as well; the MCP proposal tracker currently includes proposals related to signed capability declarations, tamper-evident audit records, asynchronous tool approvals, and signed execution records. That's an interesting signal. Because eventually the question won't simply be: "What tools can my agent use?" It will become: "What capabilities does this agent have, who authorized them, what did it actually execute, and can I prove it?" That's an entirely different level of AI engineering. The Mental Model I Want You to Remember Don't think: LLM + Tools = Agent Think: LLM + Memory + Tools + Permissions + Policies + Observability + Human approval = Production Agent Enter fullscreen mode Exit fullscreen mode The model is only one component. The system around the model is what makes an agent useful—or dangerous. Final Thought We're entering an interesting phase of AI development. The first wave of AI engineering was about: "How do I make the model smarter?" Then came: "How do I give the model access to my data?" Now we're asking: "How do I let the model take actions safely?" That is the real engineering challenge of agentic AI. Because once an AI can call your APIs, access your data, modify your code, and interact with the real world... the question isn't whether the agent can act. The question is whether you can control, verify, and audit what it does. And that's why AI-agent security may become just as important as the agents themselves.

来源报道 · 1 篇 · 以下链接均前往原始媒体

← 返回精选动态 原文

更多内容