智能体
dev.to ·
@Stephen Lincoln
AI 与现实世界之间缺失的一层
文章认为业界过度关注让 AI Agent 更聪明（更大模型、更长上下文、更强推理、更多工具与自主性），却忽视了 AI 决定行动之后的问题。当 Agent 有权部署基础设施、批准退款、查询生产数据库、采购库存、修改客户记录或防火墙规则时，难点不是能否执行，而是是否应该执行。现有系统回答的是身份问题（用户是否认证、API key 是否有效、服务是否有权限），而自主 AI 带来的是另一类问题：在特定情境下该具体动作是否应被允许。
AI推荐理由 提出 AI Agent 落地中被忽视的授权层问题：身份认证解决“是谁”，而自主行动需要判断“在特定情境下该不该执行”，涉及退款、部署、改防火墙等高风险操作，对 Agent 安全与治理有实际价值；但为观点性文章，未给出实现方案或数据。
70
正文
We're Measuring the Wrong Thing in AI Agents
Everyone seems focused on making AI agents smarter.
Bigger models.
Longer context windows.
Better reasoning.
More tools.
More autonomy.
Those things matter.
But I think we're overlooking a different question.
What happens after the AI decides to act?
Imagine an AI agent with permission to:
deploy infrastructure
approve refunds
query production databases
purchase inventory
update customer records
modify firewall rules
The challenge isn't whether the AI can perform these actions.
The challenge is whether it should perform them.
Most production systems already answer questions like:
Is this user authenticated?
Is this API key valid?
Does this service have permission?
Those are identity questions.
Autonomous AI introduces a different class of question:
Should this specific action be allowed under these specific circumstances?
Those aren't the same problem.
For example:
An AI wants to refund $8,500 to a customer.
Authentication succeeds.
The API key is valid.
The agent has permission to call Stripe.
Should the refund execute automatically?
Maybe.
Maybe not.
It depends on context.
Is this production or staging?
Is this a VIP customer?
Has fraud been detected?
Has a human approved refunds above $5,000?
Is another agent already processing the same request?
None of those questions are answered by authentication alone.
I think we're moving toward a new architectural pattern:
Intent
↓
Policy Decision
↓
Execution
Instead of allowing AI agents to directly change the real world, every high-impact action crosses a governance boundary first.
Not because we don't trust AI.
Because we need systems that businesses can trust.
I've been exploring this idea through a project called Ex.
The vision isn't to replace AI frameworks or build another agent platform.
It's to explore whether autonomous AI needs a dedicated execution governance layer that sits between agents and real-world consequences.
Think less about making AI more intelligent.
Think more about making AI safe to operate at enterprise scale.
I'm curious how engineers are thinking about this.
If you were designing AI infrastructure for the next decade:
Where would you enforce trust?
Inside the model?
Inside every agent?
At the tool/function-call layer?
In middleware?
As a centralized execution control plane?
I'd love to hear different perspectives.
ai #agents #softwareengineering #architecture #devops #security #opensource #machinelearning