Prompt injection is a confused deputy problem

A language model reads its instructions and the data it works on through the same channel, as one stream of text, and cannot reliably tell them apart. Give that model authority, such as credentials, a signed-in browser or tools that change things, and anyone who can put text in front of it can borrow that authority. Security has had a name for this failure since 1988, and the name points to the defenses that work.

First principles

A large language model (LLM) receives one sequence of tokens and predicts the next. The system prompt, the user’s request, a web page, an email and a tool result are joined into that sequence before the model runs. The role markers between them are tokens too. A provider can stop untrusted text from forging a marker, but whether the model treats what follows one as data is a habit learned in training, not a boundary anything enforces.

Structured Query Language (SQL) injection was fixed by parameterised queries: the query and the user’s values travel on separate paths, and the parser never reads a value as code. A model has no second path. Text that reads like an instruction is processed the same way whether the user typed it or a stranger planted it in a page, which Kai Greshake and colleagues named indirect prompt injection in 2023.

An agent’s power does not come from the text. It comes from the harness: the browser’s session cookies, the token for the mail application programming interface (API), the shell. A tool call runs with all of that authority, and nothing in the call records which part of the context caused it.

Three things follow.

Any text the model reads is a possible instruction. Content feeds the control flow, not only the answer.
Authority is ambient. The agent holds it the whole time, so a stranger’s request runs with the owner’s rights.
The model cannot be the check. It is the component that gets confused.

The confused deputy

Norm Hardy described the problem in Operating Systems Review in October 1988. At Tymshare, a timesharing company, a compiler held a licence to write files in its home directory so it could record usage statistics. Users could name a file for its debugging output. One user named the billing file in that directory, and the compiler, using its own licence, wrote over the billing records.

The compiler’s code was not wrong. It acted for two parties and could not say whose authority a given write used. “The fundamental problem is that the compiler runs with authority stemming from two sources,” Hardy wrote. His answer was capabilities: the caller hands over a reference that both names the output file and grants the right to write it, so the compiler can only write where its caller could.

An agent is the same deputy in a worse position. Its second source of instructions is whoever wrote the text it read. Hardy’s compiler at least knew which string was a file name; a model does not know which string is an instruction.

The United Kingdom’s National Cyber Security Centre (NCSC) draws the hard conclusion. A classic confused deputy is a bug that can be fixed. A language model is a deputy that can always be confused, so the risk can be lowered but not removed, and the NCSC advises deterministic safeguards outside the model.

The combination that turns confusion into harm

A model that only summarises a page can be made to write a misleading summary, a bounded problem. Harm needs reach. Simon Willison named the dangerous combination the lethal trifecta: access to private data, exposure to untrusted content, and a way to communicate externally. With all three, a planted instruction can tell the agent to read something private and send it out. Meta’s Agents Rule of Two widens the third property to any ability to change state, and advises that an agent hold at most two of the three in one session, or work under human approval.

Each property is a normal feature, so they are easy to collect by accident. Outbound channels are also less obvious than a send button: any request can carry data in its address, including an image a client fetches on its own.

EchoLeak, Common Vulnerabilities and Exposures (CVE) entry CVE-2025-32711 in Microsoft 365 Copilot, had all three. An email carried instructions phrased like a note to a person, which passed Microsoft’s injection classifier. Retrieval later pulled it into context beside internal data, and the model could be steered into putting that data in a link or image. Outside domains were mostly blocked, but trusted Teams and SharePoint addresses could carry it. Microsoft fixed it on its servers and reported no exploitation. The classifier missed it, and the allowlist leaked through hosts it trusted.

Why filters and warnings are probabilistic

The usual first defenses work inside the model’s channel: a system prompt telling the model to ignore instructions in content, delimiters, an injection classifier, training that ranks system instructions above data. They help. OpenAI’s instruction hierarchy work reported large robustness gains, and Microsoft’s spotlighting, which marks where untrusted text came from, cut attack success from over 50% to under 2% in its tests.

They stay probabilistic for structural reasons.

The attack space is open. An injection is any text that changes behaviour. It can be paraphrased, translated, encoded, or hidden as faint text in a screenshot, as Brave showed against Perplexity’s Comet browser. A deny list of phrases lists attacks someone already tried.
The detector is the same kind of system. A classifier that reads the content can itself be steered, and it has a miss rate.
The attacker adapts. In The Attacker Moves Second, Milad Nasr, Nicholas Carlini, Florian Tramèr and colleagues bypassed 12 published defenses, most with attack success above 90%, though most had reported near-zero rates in their own evaluations.
Rates compound. An agent reads thousands of pages and an attacker can try many times. A defense that stops 99% of attempts lets at least one of 100 independent attempts through about 63% of the time.

Willison notes that a filter catching 95% of attacks would be a failing grade in web security. The Open Worldwide Application Security Project (OWASP) ranks prompt injection first among LLM application risks and says it is unclear whether any foolproof prevention exists. A rate tells you how often you lose. It is not a boundary.

Designs that hold by construction

The alternative is a system in which it does not matter whether the model keeps instructions and data apart. A 2025 design patterns paper by Luca Beurer-Kellner and colleagues states the test: once an agent has read untrusted input, that input should have no way to trigger a consequential action.

Keep the planner away from untrusted text

Willison’s dual-model pattern (2023) splits the agent. A privileged model takes the user’s request and can call tools, but never sees untrusted content. A quarantined model reads untrusted content but has no tools. Ordinary code between them holds the quarantined output in variables, which the privileged model refers to only by name.

The CaMeL paper by Edoardo Debenedetti and colleagues at Google, Google DeepMind and the Swiss Federal Institute of Technology in Zurich (ETH Zurich), published in 2025, makes this concrete. The privileged model turns the request into a program in a restricted subset of Python before any untrusted data is read, and a custom interpreter runs it. The quarantined model only parses untrusted data into fields the program asked for. Every value carries capabilities recording where it came from and who may read it, and policies check them before each tool call, so a document can be emailed only to someone allowed to read it. The name stands for capabilities for machine learning: Hardy’s answer, applied to agents.

On the AgentDojo benchmark, CaMeL completed 77% of tasks with its guarantee against 84% undefended, using about 2.7 to 2.8 times the tokens for the median task. It leaves open misleading summaries, side channels such as timing, and data steering a choice among actions the plan already allows.

Scope capabilities and permissions

Give each task the least authority it needs: a token for one folder, read access before write, credentials that expire with the task. The NCSC passes on a rule from Willison and Baibhav Bista: when a model reads a party’s content, its privileges should drop to that party’s. Google’s design for the agent in Chrome scopes websites per task. A trusted gating function decides which sites a task may read and which it may act on, and the planner cannot add a site without that function’s approval. Google says its first version tracks only the sites a task may act on.

Confirm irreversible actions in code

Sending, paying, publishing, deleting and granting access should stop for a person. The check belongs in code keyed on the action, not in the model’s opinion of whether it looks risky, and the person should see the real arguments: recipient, amount, destination. Meta and the CaMeL authors both name the failure mode, a person who approves without reading, and the CaMeL authors tie it to fatigue from being asked too often. Confirmations work when they are rare enough to be read.

Limit egress

Decide in advance where a steered model could send data. Deny outbound requests by default and allow hosts by name. Stop the model from building addresses out of data it read; Chrome restricts addresses the model generates to known public ones. Check whether an allowed host can relay data, as EchoLeak’s could. Rendering an image from model output is an outbound request.

A browser in your signed-in session

Troy, our browser for agents, is a sharp case. A person uses it with their own accounts, and an agent attaches over the Chrome DevTools Protocol (CDP) to work inside that signed-in profile, because the pages worth automating sit behind a login. So one session can hold all three properties: the profile is private data, every page is untrusted content, and navigation and requests are outbound channels. As Brave noted, the same-origin policy, which stops one site reading another, does not constrain an assistant acting across sites with the user’s sessions.

What Troy does, from its code and docs:

The agent port is off unless Troy is started with a flag or environment variable, for that launch only. Once open, any local process gets full control of every signed-in tab, with no token, and the docs say so.
Refusals live in code the page and agent cannot reach. The address bar’s resolver refuses schemes such as javascript:, and on main the agent’s navigate tool goes through the same resolver. Page permission requests are denied except fullscreen and sanitised clipboard writes.
On the main branch, the read pipeline drops text hidden by rule, such as zero-size, transparent or white-on-white text, and never returns password values; the tool layer’s own page read also withholds them. The tool layer’s model loop waits for an explicit yes before navigate, click, fill and select, and click refuses controls that would submit a form or whose text reads like a commit, such as pay or send.

Where it stops:

The tool layer is library code the app does not use yet, and the gate is the caller’s job.
The hidden-text rules are rules, not vision: faint, low-contrast text like Brave’s still passes, and the tool layer’s page read relies on the browser’s rendered text, which does not catch white-on-white.
The system prompt’s rule that page text is data is a probabilistic defense, and the commit-word check is a heuristic that reads button text, so a consequential button with neutral wording would pass it. Only the form-submit check is structural.
page_scrape is ungated and fetches any https address through curl_reap, our Python library for web requests. That is an outbound channel, and by this post’s argument it should be gated or limited to named hosts.
An agent attached directly over CDP skips the tool layer and can run any script in any tab. There is no per-task site scope like Chrome’s.

So a browser for agents has to keep the control port closed unless asked, put refusals in the browser rather than the prompt, gate every write and send behind a person who sees real arguments, scope each task to named sites, and treat every outbound request as egress, reads included.

The strongest case against this view

Builders who rely on model-level defenses have a serious argument. Measured rates are falling fast: Anthropic’s browser evaluations went from double-digit attack success to zero for most models it tested with full safeguards, though the evaluations changed along the way (see the dated section). Architecture costs task success and tokens, and the most useful agents need all three properties: an assistant that reads your mail, browses and replies is the product. Security already lives with probabilistic controls such as spam filters. And people are confused deputies too; OpenAI says prompt injection, like scams and social engineering, is unlikely ever to be fully solved.

Much of this is right. Layers that lower the rate are worth having, and for an agent that only reads and summarises, a good classifier may be enough. Architecture fails too: confirmations get approved unread, and CaMeL leaves misleading summaries in scope. Willison objected that untrusted input plus the power to change state can do harm without private data; Meta answered that its second property covers any sensitive system, and relabelled the pairs lower risk rather than safe.

The argument falls short on compounding. Injected text costs nothing to plant and is read by every agent that passes. And the employees we trust work inside structural limits: spending caps, two-person approval, separated duties. Those are scoping and confirmation, not training alone. The defensible position is both: lower the rate with model-level defenses, and bound what one success can do with design. Never use the first in place of the second.

What to do

Draw the trifecta for each agent. List its private data, its untrusted inputs, and every way it can send or change something. If all three meet in one session, remove one or add a person.
Plan before reading. Fix the plan from the request before untrusted data arrives, and give that data only to a component without tools.
Make authority explicit. Narrow, short-lived credentials per task, and a scope the model cannot widen.
Gate side effects in code, before the tool runs, showing real arguments.
Close egress. Deny by default, allow hosts by name, and check them for relays.
Test the soft layers with adaptive attackers and repeated attempts, not a fixed list.
Log tool calls, including refused and failed ones, which the NCSC notes can reveal an attack in its early stages.

As of 2026-10-25

Measure	Value	Source
Claude for Chrome pilot, August 2025	23.6% attack success without mitigations, 11.2% with, on 123 tests across 29 scenarios	Anthropic
Anthropic browser use, November 2025	1.4% for Claude Opus 4.5 with improved safeguards (8.0% for the model alone), against an adaptive attacker given 100 attempts per environment	Anthropic
Claude in Chrome, August 2026	Stronger red-team attacks, counting attacks that reached the model: 17.6% (Opus 4.5) and 3.8% (Opus 5) before added safeguards; 0% for three models and 0.3% for one with probes and the safety classifier	Anthropic
Troy	Version 0.1.5; tool layer on main at commit 8e26c5a, not used by the app	Troy repository and docs

Vendor figures come from each vendor’s own evaluation and are not comparable across vendors or years.

What we could not verify

OpenAI’s Atlas hardening post refused our requests, so its comparison to scams comes from Fortune’s report.
No vendor attack success rate here has been reproduced independently, and Anthropic retired its earlier evaluation and replaced its grader.
We did not check that every part of Chrome’s agent design shipped as described.
We did not attack Troy. Its gaps come from reading the code and docs.

Sources

← All journal entries