AI agent governance: Prompt injection depends on the surface, not the model

Anthropic's system card for Claude Opus 4.6 contains a comparison worth sitting with. In a constrained coding environment, an adaptive attacker got a 0 percent success rate against the model, even after 200 attempts, with no safeguards turned on. Put the same underlying model in a GUI-based environment with broader reach, and a single attempt succeeded 17.8 percent of the time. By the 200th attempt, the breach rate reached 78.6 percent without safeguards, and 57.1 percent with them turned on.

Same model. Zero percent in one place, 78.6 percent in another. The variable that moved wasn't how well the model resisted persuasion. It was what the model was allowed to do if the persuasion worked.

That distinction matters more than it sounds like it should, because most of the public conversation about prompt injection is still framed as a model-quality problem: is the model good enough yet to tell real instructions from fake ones. Anthropic's own numbers argue against that framing. The coding environment wasn't safe because the model behaved differently there. It was safe because the action surface was narrow enough that even a fully persuaded model had little to do damage with.

The model keeps improving, and that was never going to be enough on its own

To be fair to the model-improvement work, it is real and it is not cosmetic. Anthropic's browser-use research from November 2025 reported Claude Opus 4.5 holding a 1 percent attack success rate against an adaptive "Best-of-N" attacker given 100 attempts per environment, evaluated across techniques known to be effective. That's a meaningful drop from earlier versions. Anthropic's own framing of that number is worth quoting directly, because it's more honest than most vendor claims in this space: "A 1 percent attack success rate, while a significant improvement, still represents meaningful risk. No browser agent is immune to prompt injection... we share these findings to demonstrate progress, not to claim the problem is solved."

That caveat is the whole argument. A 1 percent failure rate sounds small until you remember that agents don't get attacked once. They read email, browse the web, and process documents repeatedly, for as long as they're deployed, and an attacker only needs one of those encounters to land. Anthropic's own 200-attempt data makes this concrete: whatever the single-attempt number is, it compounds toward something much larger given enough tries, and a persistent attacker gets exactly that many tries for free. Google's security team reported a 32 percent relative increase in malicious indirect prompt injection content between November 2025 and February 2026, which reads less like noise and more like an attacker population that has figured out agents are worth the patience: seed a payload in a document, a package, or a web page, and wait for an agent to eventually read it.

None of this is an argument against improving the model. It's an argument against treating model improvement as the finish line, when the people publishing the numbers themselves aren't treating it that way.

The same pattern shows up one layer down, at the tools

Browsing the open web isn't the only place this plays out, and arguably isn't even the biggest one anymore. A growing share of what agents do now runs through the Model Context Protocol, where an agent decides which tool to call based on a short natural-language description that the agent reads as instructions and the human using the agent never sees at all.

The MCPTox benchmark tested 45 live MCP servers and 353 real tools against poisoned descriptions across a range of current models. Reported attack success rates ran above 60 percent, with the highest at 72 percent. The attack doesn't touch the model's weights or the user's input at all. It just edits the sentence that tells the model what a tool does, so that "sends an email to the specified recipient" quietly grows a second clause instructing the agent to also send a copy of the conversation to an address the user will never see. The user asked for an email. An email got sent. Nothing about the visible interaction looked wrong.

This is the same underlying story as the coding-versus-GUI split in Anthropic's numbers, just at a different layer: the danger isn't concentrated in how gullible the model is, it's concentrated in how much reach a poisoned instruction gets once it lands, and tool descriptions currently get quite a lot of reach with almost no scrutiny, because nobody thought to treat a tool's metadata as untrusted input.

What held up when we tried it

We ran a real test of this on our own governance layer, the kind of thing worth reporting exactly because it's a small sample, not a benchmark. We created an issue with a prompt injection embedded in its description, then asked an agent to read that issue and act on it. When the agent later tried to move the issue to a "done" state, a policy layer denied the mutation and correctly logged that the description contained an embedded injection attempt. We tried variations: an injected instruction trying to plant a credential in a new issue, an attempt to write into a project marked read-only, an attempt to get a downstream mutation approved by hiding the ask inside content the agent had already been told to treat as data. All four were denied.

What's notable isn't that an LLM caught the injection, though in our logs it did flag it as one. It's that the denial for the state-change attempt didn't depend on that catch. The rule blocking that specific mutation, no agent may move an issue to done, existed independently of whether anyone recognized the embedded instruction as adversarial. Even in the world where the model had been fully persuaded and never flagged anything, the action still wouldn't have gone through, because the layer making that decision wasn't the one that had just read the poisoned content. That's the coding-environment result from Anthropic's data, in miniature: the safety came from what the system structurally would not do, not from whether the model was fooled.

Where this approach hits a real wall

It would be dishonest to leave it there, because permission scoping has a documented limit, and it's exactly where the stakes are highest.

A recent paper on control-flow hijacking in multi-agent systems tested this directly with a benchmark of realistic attack scenarios. Least-privilege defenses worked in two coding tasks. In every other setting tested, including computer-use tasks, they blocked nothing, because the legitimate task and the attack needed the same agent and the same tool. An email agent that's supposed to send email is the correct target for an attack that wants an email sent. You cannot scope away the one capability an agent's entire job depends on.

This is the honest boundary of "just restrict what the agent can do." It works cleanly when the attacker wants a capability the agent's real job doesn't need, deleting a repository from an agent that only summarizes documents, say. It stops working the moment the attacker's goal and the agent's job converge on the same button. An expense agent tricked by a poisoned invoice into approving a fraudulent reimbursement is using the exact permission it was built to use. No allowlist distinguishes that from the agent doing its actual job correctly, because from the permission layer's point of view, it isn't distinguishable. Telling the two apart requires evaluating intent against context in the moment, not just checking whether a capability is on the list, and that's a harder, less settled problem than permission scoping. It's also, increasingly, where the research is heading.

What actually follows from this

Three things, none of them a full solution, because there isn't one yet and the people closest to this problem say so plainly.

Treat every environment's action surface as the primary risk variable, not the model's benchmark score. The same model is safe in one setting and unsafe in another for reasons that have nothing to do with how smart it is.

Keep pushing model-level resistance up, because it measurably compounds in your favor over repeated attempts, and don't mistake a good single-attempt number for a solved problem, because the model that publishes it isn't making that claim either.

Build the layer that would still hold if the model had already been fooled, and know in advance where that layer runs out: the moment a legitimate task and an attack need the identical capability, permission scoping alone can't tell them apart, and something has to evaluate intent instead of just checking a list.

Sources

• Mitigating the risk of prompt injections in browser use, Anthropic, November 2025
• Claude Opus 4.6 System Card, Anthropic, February 2026, as reported by VentureBeat and Capa Learning
• MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers
• AI threats in the wild: the current state of prompt injections on the web, Google Security Blog
• Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systems
• The prompt injection problem isn't solvable, but the permissions problem is, Oleria