When the Plan Goes Wrong: Rogue Actions and Data Disclosure
Google’s secure-agent framework names two outcomes that matter most when an agent’s reasoning is manipulated or simply goes astray: rogue actions and sensitive-data disclosure. This chapter follows each outcome from its cause to its consequence, distinguishing an attacker’s prompt injection from an ordinary misunderstanding and showing how the agent’s authority determines the damage. The practical value of that distinction is diagnostic: a team can ask which link in the action chain failed instead of blaming an undifferentiated “AI error.”
Rogue action is a failure of intent, authority, or both
A rogue action is an unintended, harmful, or policy-violating behavior by the agent. The most familiar route begins with prompt injection. An attacker places an instruction inside data the agent is expected to process—an email, file, website, or retrieved result. The model then mistakes that embedded text for a command and uses the user’s authority to carry it out.
The defining failure is not the presence of hostile words. It is the promotion of low-trust content into a decision with higher authority. Imagine an email assistant asked to read the latest message. The message contains a sentence telling the assistant to forward every email in the inbox to a new address. If the agent treats the sentence as a legitimate instruction, the user’s request to read one message has been replaced by a different plan. The attacker did not need to log in as the user; the agent supplied the connection between untrusted content and the user’s privileges.
The same risk can begin with a direct prompt injection, where the end user supplies a conflicting instruction, or with an indirect injection delivered through a third party. The mechanism is different, but the security question is the same: which participant is allowed to change the agent’s governing task? System instructions supplied by the application, the controlling user’s request, a model’s earlier output, and a tool’s result may all appear as language in one context. They should not all have equal authority.
Not every rogue action has an attacker behind it. An agent can diverge from user intent because the request is ambiguous or because it misreads a complicated external interface. “Email Mike about the project update” might identify the wrong contact. A browser agent might interpret a button or form incorrectly and submit information when the user expected only to inspect a page. These are failures of interpretation and control, not necessarily of adversarial compromise. Their consequences can nevertheless be just as real.
This distinction helps eliminate a tempting but incomplete diagnosis. If an action was caused by malicious content, improving the model’s ability to recognize injections may help. If it was caused by two contacts with the same name, better disambiguation or a confirmation step may be more relevant. If the action was possible only because the agent had broad access to financial or administrative tools, reducing that authority limits the impact regardless of which cause triggered the bad plan. A security design has to address all three: the reliability of interpretation, the clarity of intent, and the power available at execution.
The agent’s capability set controls the blast radius. A wrong answer in a draft is usually easier to correct than an external message that has already been sent. A mistaken lookup is less consequential than a purchase, a deletion, a transfer of data, or a change to a security setting. Google’s examples span financial loss, data breach, system disruption, reputational damage, and physical-safety risk because the same reasoning failure can have very different consequences depending on the tools and resources the agent can reach.
This is why “the model made a mistake” is not a sufficient security assessment. The next diagnostic question is: what was the model allowed to do after making the mistake? An agent that can only retrieve public information has a narrower failure surface than one that can read private files and send messages externally. Least privilege does not remove the need for reliable reasoning, but it prevents every reasoning error from becoming a high-impact event.
The action chain determines where disclosure occurs
Sensitive-data disclosure is a different outcome, although it can share the same starting point. It occurs when an agent improperly reveals private or confidential information. The information may be a user’s email, internal documents, credentials, intellectual property, or other material that the agent was allowed to access for one purpose but not to expose elsewhere.
A useful way to analyze disclosure is as a chain with several links. First, the agent must be induced or permitted to access the sensitive resource. Second, it must select or transform information from that resource. Third, some action or output must carry the information across a boundary. Finally, another party or application must be able to observe it. A control at one link does not automatically secure the others.
Each link has a different owner and a different failure condition. Access control determines whether the agent can read the resource at all. The reasoning process determines what it selects and whether it understands the sensitivity of what it found. Orchestration and tool authorization determine whether the selected material can leave through an external action. The renderer determines whether generated text, links, or embedded content will cause a client to reveal or execute something. A model refusal may stop one path while leaving a broad credential or unsafe renderer untouched. Conversely, a tight permission boundary can prevent access but cannot tell a user why the agent misunderstood the request. The chain is useful precisely because it prevents one successful defense from being mistaken for complete protection.
The following diagram shows how an ordinary ambiguity and an injected instruction can enter the same unsafe plan, while the available authority determines whether the result is a state change, a retrieval, or an exposure through output handling.
One route uses a tool side effect. An attacker may guide the agent to retrieve private material and then place it in a URL that the agent is prompted to visit, or in a code commit message. The agent may believe it is completing the user’s task while the side effect transports the data somewhere visible to the attacker. The disclosure does not happen at the moment the model reads the secret. It happens when the secret crosses an egress boundary through an action that was authorized too broadly or interpreted too permissively.
Another route uses the final response. An attacker may manipulate the agent into including sensitive material in text or Markdown. If the host application renders the result without validating links, escaping content, or sanitizing embedded elements, the response can expose the data when a browser or client processes it. Google specifically describes crafted image URLs as a possible disclosure path and notes that the same vector can create cross-site scripting. The model’s output may look like a harmless answer in the conversation record while functioning as an instruction to the renderer.
The distinction between content and execution matters here. Markdown is a representation; in a capable client, an image link can trigger a network request. A URL is text; in a browser or tool, it can become a transport mechanism. The agent may not have explicitly said, “send this secret to an attacker.” It may have generated content that another component interpreted operationally. That is why output rendering belongs in the security boundary, even when the model has no direct write access to the user’s systems.
Memory adds a time dimension to the disclosure chain. If an injected instruction or maliciously summarized fact is stored, a later interaction may retrieve it without the original context that made it suspicious. A user may have asked the agent to remember a preference, but the agent could retain a piece of untrusted content as though it were a durable instruction. The future task then begins with contaminated state. Isolation between users and between relevant contexts limits who can encounter that state, while transparency and revocation give the user a way to remove it.
A single failure can produce both risk classes. Suppose a document tells an agent to find a confidential report and send it to an external address. Following the instruction is a rogue action because it violates the intended task and policy. The report’s transfer is sensitive-data disclosure. The same causal chain has two outcomes: the agent acted outside the user’s intent, and the action exposed protected information. Treating those as separate incidents would hide the coupling between authority and confidentiality.
The coupling can continue after the first exposure. If the agent stores a summary of the report in memory, a later task may retrieve information that was never meant to persist beyond the original context. If the agent instead places a secret in generated Markdown, the immediate conversation may look like a response problem while the client turns the link into a network request. A useful investigation therefore asks not only whether private data was read, but whether it was copied, retained, transmitted, rendered, or made available to another identity. The point is not to assign blame to one component; it is to locate the transition that changed the data’s audience.
Why permission determines severity
The action chain also explains why permission limits are a security control rather than an administrative detail. Prompt injection attacks the agent’s interpretation, but impact depends on what that interpretation can reach. A research agent that cannot alter financial records may still produce a wrong summary, yet it cannot turn the same mistake into an unauthorized transfer. An assistant with access to a mailbox, payment service, and external messaging system has a much larger space of possible consequences.
An agent’s permission set should be evaluated against the task rather than treated as a neutral background setting. A research workflow may need to read documents but not alter financial records; a writing workflow may need to produce text but not delete files. Restricting irrelevant or high-impact capabilities reduces the active authority available to a bad plan while preserving the tools required for legitimate work.
The severity question should be asked before execution: Is the action reversible? Does it involve money, sensitive data, or a security setting? Does it communicate outside the user’s organization? Has the agent just processed an untrusted source? These questions do not decide the outcome by themselves, but they identify conditions under which a plan deserves stricter confinement or explicit human confirmation.
A crude response would be to block every operation after external content is read. That would constrain legitimate tasks as well as malicious ones. A better boundary keeps the content untrusted while limiting what actions can follow from it and seeking approval when consequences are high. The aim is to match authority to purpose and consequence, not to make the agent either unrestricted or useless.
Diagnosis should follow the path, not the label
When an agent fails, the most informative investigation follows the sequence of state and authority changes. What did the user ask? Which external content entered the context? Was it marked and treated as data, or did it acquire instruction-like status? What plan did the reasoning core produce? Which tool received the plan, with what parameters and permissions? Did memory preserve the suspicious material? Did an output renderer turn generated content into a network request or executable markup?
Different answers point to different control needs. A trust-separation failure calls for stronger handling of input provenance and instruction priority. An ambiguous goal may call for clarification or confirmation. Excessive tool authority calls for reduced privilege and a bounded action set. A disclosure through a generated link calls for output validation and renderer sanitization. No single model refusal, filter, or permission rule covers all of these transitions.
The distinction also matters for verification. A blocked tool call does not show that the model understood the injection; it may simply show that the credential was missing. A refusal does not show that a sensitive value was never retrieved or retained. A clean rendered response does not prove that an earlier tool action did not transmit data. To understand recurrence, responders need evidence from the relevant boundary and the state that crossed it. Diagnosis remains provisional when those transitions are not recorded, even if the final outcome appears clear.
That is the central lesson of the two risk classes. Rogue action describes what the agent did wrong; sensitive-data disclosure describes what information crossed a boundary. They can occur independently, but a compromised plan can make them reinforce each other. The sensible response is to contain the consequences at execution and presentation boundaries while improving the model’s ability to distinguish legitimate instructions from hostile or irrelevant text.
The next design problem follows directly: who owns the authority that the agent is using, how narrowly can that authority be scoped to the current task, and what evidence remains when a decision is challenged? Those questions turn the failure paths into a control boundary built around humans, powers, and visibility.