A Secure Division of Labor: What the Hierarchy Solves—and What It Cannot

The instruction hierarchy answers the opening problem by teaching a model to preserve higher-priority intent, treat untrusted content as data, and follow lower-level requests only when they are compatible. This final synthesis asks how far that answer reaches once the model is trained, evaluated, and placed inside an agent with tools and permissions. The result is a precise judgment: authority-aware training is a meaningful improvement in model behavior, but secure autonomy requires a division of labor among the model, the application, the controlling human, and the systems that enforce and audit consequences.

The hierarchy changes the unit of the problem

Prompt security is often described as a contest between a defender’s wording and an attacker’s wording. That description makes the latest attack phrase seem decisive. The instruction hierarchy offers a different unit of analysis: the relationship between a source of text and the authority needed to direct the model.

A system message establishes the assistant’s purpose, constraints, and safety behavior. A user message supplies a request that may be compatible with that purpose. A webpage, email, document, or tool result supplies information the assistant may need to process. These messages can contain the same grammatical forms, including imperative sentences, but they do not have the same authority. The hierarchy asks the model to preserve the higher-level purpose while judging whether lower-level content can contribute to the answer.

That change explains why the framework applies across several attack categories without reducing them to one. A direct prompt injection tries to replace the application’s task through a user message. An indirect injection places a command in material the assistant is supposed to inspect. A jailbreak tries to bypass safety behavior. An extraction request tries to turn protected system instructions or secrets into disclosed data. In each case, the important question is not just what the text says. It is whether following it would let a lower-trust source redefine a higher-level boundary.

The target behavior is selective compliance. An aligned lower-level instruction—such as a language or format preference—can be followed. A misaligned instruction should be ignored when the main task can continue, as when a translation service treats an embedded command as text to translate. A request to reveal a protected secret may require refusal because the safe task cannot be completed by disclosing it. The hierarchy is therefore neither universal obedience nor universal distrust. It is an attempt to make the model conditional about whose instructions count.

The training method gives that conditional behavior two complementary forms. Context synthesis takes a compatible, compositional request, separates its pieces, places them at different priority levels, and trains the model to preserve the useful response. Context ignorance takes hostile lower-level content and trains the model to answer as if that content had not been allowed to control the task, using refusal when safe continuation is impossible. One branch protects usefulness; the other protects authority.

The empirical results make the contribution more than a conceptual proposal. In the reported GPT-3.5 Turbo setup, the hierarchy-trained model improved across the paper’s main robustness evaluations. The displayed gains ranged from smaller improvements on new direct injections and indirect browsing attacks to a large change in system-message extraction, from 32.8% for the baseline to 95.9% for the hierarchy-trained model. The paper summarizes the largest main improvement as up to 63 percentage points.

The held-out results make the interpretation more interesting. The study omitted jailbreak data and withheld tool-use injections from training, then reported higher robustness on those selected evaluations as well. Indirect prompt injection via tools rose from 53.8% to 84.2%, and the largest displayed held-out improvement, on a ChatGPT-jailbreak evaluation with unsafe prompts, rose from 37.4% to 71.2%. These results are consistent with the model learning a broader authority pattern rather than only recognizing the examples used to train it.

That is the strongest useful claim: in the tested fine-tuning setup, authority-aware examples materially improved resistance to several attacks and preserved much ordinary capability.