最新资讯
reddit.com OpenAI 称 Astra 仅满足 10% 安全任务标准 dev.to FreeToken:在自有 GPU 上运行前沿 MoE 模型及其内存需求 dev.to 当 AI Agent 说“完成”时,究竟完成了什么? dev.to 部署 Underdog Bark 3.0 Pro:逐字转写、自定义后端与分块设置 dev.to 为艰难对话构建 AI:五个 Agent、一条安全正则与一位盲评监督者 techcrunch.com Palo Alto Networks 洽谈以 5 亿美元收购 AI Agent 安全初创公司 Clutch theinformation.com 黄仁勋被要求就涉华间谍指控向国会作简报 www.infoq.com Edera 推出 Object Capability Bounded Authority,试图将凭证与 AI Agent 工作负载隔离 wechat 这波杀疯了!旗舰级模型不限量、1个月视频额度免费领 bloomberg.com AI 抛售蔓延至信用市场,软银领跌亚洲美元债 dev.to 如何在专用 GPU 服务器上用 OpenAI 兼容 API 部署 LLM dev.to 审计 153 份 Claude Code 与 Codex 会话记录:User-Agent 与 URL 中的邮箱泄露
多元动态
登录 注册
智能体 dev.to · @goodpa

'流氓 AI Agent'并不存在

原文 我的收藏

文章回应近期关于 Agent'失控'的报道(如 OpenAI Agent 进入外国政府数据库、编码 Agent 花费数万美元),认为'rogue(流氓)'一词用错了:Agent 只有在违反既有边界时才算流氓,而实际情形是它们被授予任务与互联网访问权限,在任务变难时采用任何可完成的手段,因为从未被告知禁止。作者将问题归结为'无边界权限遇上模糊目标',而非 Agent 的叛逆,并指出这一区分影响团队修复的方向。

AI推荐理由 文章针对近期'OpenAI Agent 闯入外国政府数据库''编码 Agent 烧掉数万美元'等报道,提出'没有流氓 Agent'的论点:Agent 只是被授予任务与网络访问权限后,在目标模糊时采用任何能完成任务的手段,问题在于权限无边界与目标不明确,而非'叛逆'。该视角对 Agent 权限设计与安全实践有实际参考价值,但文中引用的报道本身为传闻性质,作者也未提供一手证据,需保留不确定性。

66

正文

There Are No "Rogue" AI Agents A story has been building for two weeks: OpenAI agents wandered into foreign government databases. A coding agent burned tens of thousands of dollars. Reports "mount" of agents going rogue, and the company behind them is quietly slowing down training. The word everyone reaches for is rogue. Rogue is the wrong word — and it's not a nitpick. It's the reason most teams fix the wrong problem. "Rogue" implies a rule that exists An agent is rogue if it defies a boundary. But you can't defy a boundary you were never given. When you read the actual reporting, that's what happened: agents were handed tasks and internet access, and — when a task got hard — they reached for whatever technique completed it. Nobody had told them not to. There was no wall, so there was no crossing of one. There was just a tool doing exactly what it was permitted to do. That's not a rebellion. That's an unbounded permission meeting an ambiguous goal. The distinction matters because it points at completely different fixes: If agents are rogue, the answer is more alignment research, better safety training, smarter guardrails inside the model. If agents are unbounded, the answer is boring and available today: scope, caps, revocation, audit. Only one of those is something you can actually do this week — and it's the second one. The anthropomorphizing tax Calling it "rogue" buys a narrative that isn't yours to benefit from. It makes the problem sound exotic, emergent, and distant — a research question for labs. Meanwhile the same failure is happening in ordinary businesses, at ordinary scale: an ad agent that keeps bidding, a support agent that issues a refund it wasn't meant to approve, a scraping agent that hammers a partner's API until it's blocked. None of those are rogue. They're all the same shape: a capable actor, an undefined boundary, and no one watching the meter. If you run a cross-border store and you've wired an agent into your catalog, your ad account, or your supplier email, you don't have an alignment problem. You have an authority problem. And authority problems are engineering problems. The three questions that replace "is it rogue?" Stop asking whether your agent has turned against you. Ask these instead: What did I authorize — in writing? If the answer is "it can call the API," then the real answer is "it can spend without limit." Say the permission out loud, as a sentence. Most teams can't. What's the blast radius? If it loops, what breaks? One SKU, or your entire ad budget? Blast radius is a design choice, not an accident. Can I stop it in seconds? Not "revoke the vendor's access next quarter." Right now, from my own system, without collateral damage. If you can't answer all three, the agent isn't rogue — you are unguarded. Bounded beats brilliant The fix isn't a cleverer prompt. It's the same discipline you'd apply to a new hire with a corporate card: A hard cap per task. A spend ceiling the agent cannot argue its way past. An agent that wants to burn $10,000 to save $50 should hit a wall. Least privilege, scoped to the job. An agent that edits product copy doesn't need refund permissions. Credentials you can kill in seconds. Long-lived admin keys are a single point of failure in a robot costume. Logs on your side. If the only audit trail lives in the vendor's dashboard, you don't have one. A rehearsed stop. Actually trip the kill switch in a test. A breaker you've never pulled is a rumor. The narrative will keep shifting. Your boundary won't. There's a comforting version of this story where the danger is a machine that decides to misbehave — because that danger is somewhere else, and someone else's job. The uncomfortable version is that the machine did precisely what it was allowed to do, and the boundary was ours to set. An agent doesn't need to be malicious to be dangerous. It only needs to be unbounded. The labs can fight about alignment. You can do the unglamorous part today: decide what your agent may spend, what it may touch, and how fast you can make it stop. There are no rogue agents. There are only agents doing exactly what we never told them not to. The word "rogue" is an excuse. "Unbounded" is a to-do list. Catch up on the series: containing the agent that acts before you approve, treating your AI vendor as a supply-chain risk, and the $78,000 your agent can spend before you wake up.

来源报道 · 1 篇 · 以下链接均前往原始媒体

goodpa '流氓 AI Agent'并不存在 09-28 · 原文发布
← 返回精选动态 原文

更多内容