The Cache Is Part of the Context

Headroom’s cache-aware design addresses a problem that appears only after compression enters a repeated agent loop: the bytes saved in one request are not the only bytes that matter. This chapter follows a warm turn, a changed-prefix miss, and a genuinely cold turn to show how Headroom separates the stable cache hot zone from the newly compressible live zone. The goal is to help you choose between cache mode and token mode by looking at session economics, not at the largest per-request reduction.

A provider’s prompt cache does not remember the intention behind a message. It remembers a sequence of bytes or tokens that can be reused when the next request begins the same way. Headroom therefore has to answer two different questions: which new content can be reduced, and which previous content must be replayed exactly because the provider has already paid to process it.

A provider caches bytes, not intentions

In a long agent session, the beginning of a request often contains stable instructions, tool definitions, project rules, and earlier turns. The end contains the current question and the newest tool output. The stable beginning is the cache hot zone: rewriting it can invalidate the provider’s reusable prefix. The changing end is the live zone: it has not yet been forwarded for this turn and is the natural place to apply new compression.

This division is not just a convenient metaphor. Imagine a coding agent that has already sent its system instructions, tool contracts, project context, and several earlier results. On the next turn, the user asks for a new search. Headroom can keep the earlier forwarded messages byte-identical and compress the new search result. The provider can then reuse the old prefix while processing only the new suffix. If Headroom instead recompresses an earlier message—even into a smaller form—the provider may see a different prefix and have to rebuild attention states that were already cached.

The current proxy’s cache mode is built around this trade-off. It compresses the newest delta and forwards prior turns faithfully. Token mode gives priority to raw token removal and may revisit earlier turns, accepting some cache instability in exchange for more visible reduction. Cache mode may show little or no direct “tokens saved” in the stable prefix, but it can still produce the larger session-level benefit when cached input is substantially cheaper than fresh input.

This is why an agent’s context cannot be evaluated as a series of independent snapshots. The value of a transformation depends on whether its output becomes part of the next request’s stable prefix, whether the provider actually cached it, and whether the client sends back the form that was previously forwarded.

How a warm turn works

Headroom’s session engine follows a three-part pattern.

Prepare. It uses local compression state and, on a proxy path, provider-reported cache usage to determine how many leading messages can be treated as frozen. Previously computed compressed bytes can be swapped into the working copy, so the pipeline sees the same form that was sent before. The original caller’s message list is not mutated.

Process. ContentRouter examines the unfrozen live zone. It can protect instructions, cache-marked blocks, exact file reads, and other sensitive content while compressing eligible new tool output. The stable prefix remains outside the mutation path. This is where the content-specific logic from earlier chapters operates, but now under a positional and stateful constraint.

Finalize.