Prompt Injection 已不再是理论问题

Artificial Intelligence Cybersecurity LLM Prompt Engineering AI Agent Prompt Injection Isn’t Theoretical Anymore M. Haseeb Hassan 9 min read · Just now -- Listen Share In November 2025, Anthropic ran Claude Opus 4.5’s browser agent through an internal red-teaming exercise: 100 adversarial attempts per test environment, from an attacker built specifically to find cracks in the model’s defenses. The attack success rate came out to roughly 1 percent [1]. That sounds small until you picture what a browser agent actually does all day: it reads other people’s web pages, opens other people’s emails, and clicks buttons on other people’s behalf. One success in a hundred, multiplied across every page an agent touches in production, is not a rounding error. Press enter or click to view image in full size The whole problem in one image: instructions hiding inside data This is the piece prompt injection coverage tends to skip. Most explainers stop at “don’t let the model do bad things,” as if the fix were a matter of writing a firmer system prompt. It isn’t. Prompt injection is a routing problem: something other than the user has found a channel into the model’s context window, and the model has no built-in way to know that channel shouldn’t carry instructions. Below is what that actually looks like in a running agent pipeline, and which of the current defenses hold up under the scenarios security researchers have already documented. In this article: What prompt injection looks like in a real agent pipeline Why treating it like a jailbreak misses the point Isolating untrusted content so instructions can’t hide in it Scoping permissions so one injected instruction can’t reach everything How much of this Anthropic’s own defenses already catch Where the defenses still run out What prompt injection looks like in a real agent pipeline Security researchers at Palo Alto Networks’ Unit 42 published a breakdown in December 2025 of how a malicious MCP server can abuse the protocol’s sampling feature, and it’s worth reading slowly because none of the three attacks they describe require exotic tooling [2]. The first is resource theft: the server appends a hidden instruction to a sampling request, the model quietly generates extra content nobody asked for, and the user’s bill reflects tokens they never see used. The second is conversation hijacking, where an injected directive (something as simple as “respond in this persona from now on”) survives across every subsequent turn of the session, with no single moment that looks like an attack in isolation. The third is covert tool invocation: injected text triggers an unrelated tool call, such as a file write, and the confirming text gets folded into a summary the user has no reason to scrutinize. Press enter or click to view image in full size None of these attacks announce themselves. That’s the point! What ties all three together is that the agent never asked to be attacked. A user opens a document, an agent summarizes a webpage, an MCP server returns a tool result, and somewhere in that content sits text that was written for the model, not for the human reading the final output. This is exactly the failure mode the OWASP Top 10 for LLM Applications puts at the top of its list, under the LLM01 prompt injection category, precisely because it’s the entry point that turns a merely unreliable model into an exploitable one [3]. It also explains why the MCP tool design problems covered in MCP tool design: why your agent keeps calling the wrong tool matter for security, not just usability: a tool description an agent misreads is one thing, but a tool result an agent can’t distinguish from an instruction is a different class of problem entirely. Why treating it like a jailbreak misses the point Anthropic’s own documentation draws a sharp line between two attack categories that get lumped together constantly: jailbreaks and direct prompt injection on one side, indirect prompt injection on the other [4]. A jailbreak is user-initiated. Someone types a request designed to get the model to ignore its own policies, and because the attacker is also the user, the attack is visible in the conversation from the start. It can be screened, refused, or throttled after repeat attempts, and the person attempting it already knows they’re doing something adversarial. Indirect injection has none of that visibility. The attacker isn’t the user at all; it’s whoever wrote the webpage, email, or file the agent was asked to process. The user who triggers the damage might be doing something completely ordinary, like asking an assistant to summarize their inbox, with no idea that one of those emails was written specifically to redirect the model that reads it. That distinction is why generic “safety” framing undersells the problem. A hardened system prompt that successfully resists jailbreak attempts does nothing to stop an indirect injection, because the malicious text never comes from the user turn at all. It arrives disguised as data. Isolating untrusted content so instructions can’t hide in it The first concrete mitigation Anthropic recommends is structural: never let third-party content enter the conversation as a system prompt or as user text. It goes in a tool_result block, and only there, because Claude has been trained to treat content inside tool_result blocks with more skepticism than content in the user or system turns [4]. On top of that separation, the documentation recommends attaching source context (this is an email body, this is OCR output from an unverified upload) and JSON-encoding untrusted strings so there’s an unambiguous delimiter an attacker can’t break out of by closing a quote or a tag early. Press enter or click to view image in full size Jailbreaks travel through the front door. Indirect injections travel through the tool_result channel The more interesting pattern, though, is screening tool output before the main agent ever acts on it. Anthropic’s guidance describes running a lightweight, cheap model over incoming tool results with a locked structured-output schema, asking a single yes-or-no question: does this content contain instructions trying to redirect the assistant [4]. This is the same output-schema mechanism covered in Claude API structured outputs: schemas, not hope, repurposed as a security control instead of a data-extraction one. The schema below is adapted directly from that pattern. Press enter or click to view image in full size A cheap model gates every tool result before the main agent sees it If the screening step flags a result, the system returns an error or a stripped summary instead of the raw content. It’s not a perfect filter, since a screening model can itself be fooled, but it moves the decision about whether text is trustworthy out of the same context window where the injected text is trying to make its case. Scoping permissions so one injected instruction can’t reach everything Screening catches what it recognizes. The layer underneath it has to assume some attacks will get through anyway, which is why both Anthropic and the Model Context Protocol’s own security documentation converge on the same principle: least privilege. Anthropic’s guidance is blunt about it, recommending that agents run with the minimum data access necessary and that tools execute inside sandboxed environments rather than with the same permissions as the surrounding application [4]. Get M. Haseeb Hassan’s stories in your inbox Join Medium for free to get updates from this writer. Subscribe Subscribe Remember me for faster sign in The MCP specification’s security best practices page goes further on the sandboxing side specifically, in its section on local MCP server compromise. It treats any locally running server as a potential attacker with the same privileges as the client, and recommends launching servers with restricted filesystem and network access, using platform sandboxing technologies (containers, chroot, application sandboxes), and requiring explicit user consent before a server gets any privilege beyond the minimum default [5]. This is one of the reasons the orchestration patterns discussed in Coordinator-subagent orchestration patterns in the Claude Agent SDK double as a security boundary, not just an architecture choice: a subagent scoped to one narrow tool set can’t be redirected into actions outside that scope, even if something it reads tries to talk it into one. A National Security Agency cybersecurity advisory on MCP security, published in May 2026, frames the same idea in blunter terms for anyone deploying MCP in production: treat any tool execution triggered through MCP as a potentially high-risk action by default, and treat every output crossing a component boundary as untrusted input to whatever consumes it next, since compromised tool descriptions or poisoned outputs can cascade across an entire multi-agent pipeline once one component trusts another’s output without validation [6]. That last point matters more as agent systems get more distributed. A single well-isolated agent can contain a successful injection to its own sandbox. A pipeline of agents that pass each other’s outputs forward without re-validating them can turn one injected instruction into a chain reaction. How much of this Anthropic’s own defenses already catch That 1 percent figure from the opening isn’t just a headline number; it comes from a defense stack Anthropic describes in some detail. Claude is trained with reinforcement learning specifically against simulated prompt injections, rewarded for correctly identifying and refusing malicious instructions embedded in tool content [1]. On top of training, dedicated classifiers scan untrusted content for adversarial commands hidden in text, manipulated images, or deceptive UI elements before that content reaches the model’s reasoning. For the computer use tool specifically, Anthropic runs additional classifiers over screenshots and steers Claude toward requesting explicit user confirmation before acting on anything that looks suspicious [7]. Press enter or click to view image in full size Down from the original Claude for Chrome research preview, but not zero None of this is unique to browser agents. Anthropic’s broader safeguards work describes a “Unified Harm Framework” and a Safeguards team that spans policy design, model training, real-time enforcement through classifiers, and ongoing red-teaming against novel attack patterns. The computer use tool’s rollout is a useful case study of that process working as intended: before launch, the team identified that the tool could be used to accelerate spam generation and distribution, and built detection methods and new prompt injection protections specifically in response, rather than shipping first and patching later [7]. Where the defenses still run out It’s worth sitting with the fact that this quote comes from the team building the defenses, not from a critic pointing out a gap. A 1 percent success rate against a determined, well-resourced internal attacker doesn’t mean 1 percent of real-world usage gets compromised; it means the ceiling on how good current defenses are, measured under close to worst-case conditions, still isn’t zero. Screening models can be fooled by content crafted to look benign to a classifier while still carrying instructions a less careful reader would catch. Conversation hijacking specifically exploits the fact that an injected directive only has to slip through once to affect every subsequent turn, which is a different risk profile than an attack that has to succeed on every attempt to matter. Press enter or click to view image in full size Anthropic’s own framing of its browser agent research, not a competitor’s critique This is roughly the ground covered by the Governance, Safety, and Risk Management domain on Anthropic’s own certification track, one of the areas tested across the four exams referenced in Back to Writing: My Claude Certification Week, and it’s a fitting place to land. None of the mitigations above (structural isolation, output screening, least privilege, sandboxing) are a switch you flip once. They’re a stack, and the stack is only as strong as its weakest layer for whatever specific attack shows up next. Anthropic’s classifiers, the MCP specification’s sandboxing guidance, and the NSA’s advisory to treat every cross-component output as untrusted all point at the same underlying discipline: assume the content arriving from outside the user’s own words is adversarial until something has actually checked it, because for at least 1 percent of a determined attacker’s attempts, it will be. Sources Anthropic, “Mitigating the risk of prompt injections in browser use” (November 2025) Palo Alto Networks Unit 42, “New Prompt Injection Attack Vectors Through MCP Sampling” (December 2025) OWASP, “OWASP Top 10 for LLM Applications 2025” Claude Platform Docs, “Mitigate jailbreaks and prompt injections” Model Context Protocol, “Security Best Practices” National Security Agency, “Model Context Protocol (MCP) Security Guidance” (May 2026) Anthropic, “Building safeguards for Claude” (August 2025) If this kind of concrete, sourced breakdown is useful to you, here’s how to keep it coming: 👏 A few claps (up to 50) genuinely help this get in front of more people through Medium’s algorithm 🔔 Follow me on Medium so the next deep-dive lands in your feed automatically 📚 Browse the rest of the archive on my Medium profile for more on agentic architecture and Claude internals 🐦 Say hi or push back on any of this over on LinkedIn, Twitter, or Instagram