CCR: Making Compression Recoverable

CCR—Compress-Cache-Retrieve—is the mechanism that turns Headroom’s most aggressive reductions from a one-way deletion into a conditional recovery system. This chapter follows the lifecycle of a compressed result: how Headroom stores the original, places a marker in the model’s context, supplies a retrieval route, and handles expiry or missing state. The practical judgment is whether a compact result is genuinely recoverable in the integration where it is being used, not whether the word “reversible” appears in its description.

The distinction matters because the model usually does not need every byte of a large tool result on every turn. It may need only the error records, relevant search matches, or representative rows. But the model cannot know in advance whether a discarded detail will become important later. CCR separates the small working view from the larger backing record so the first request can stay compact without declaring the omitted information permanently irrelevant.

Compression creates a second representation

The basic CCR sequence has four parts.

First, a content-specific compressor produces a smaller representation. That representation may be a selected set of JSON rows, a compacted log, an abbreviated search result, or another form supported by the route. Second, Headroom stores the original content in a local compression store and associates it with a hash. Third, the visible result carries a marker that tells the model which stored entry backs it. Fourth, the proxy can make a provider-appropriate headroom_retrieve tool available so the model has a way to request the original.

The marker is not the original data and does not make the model omniscient. It is a pointer. Its usefulness depends on the store containing the corresponding entry, the marker identifying the correct entry, and the model or handler having a route that understands the pointer. When those conditions hold, the model can work from the compact view and retrieve more only when the task requires it.

On supported proxy paths, Headroom can handle the retrieval as part of the request loop. The model emits a call to headroom_retrieve with the hash. The response handler recognizes that the call belongs to Headroom, looks up the original locally, adds the retrieved content to the conversation, and continues the provider interaction. The application can therefore receive the model’s eventual answer without having to implement a second provider conversation for every retrieval.

sequenceDiagram participant T as Tool participant H as Headroom participant S as CCR store participant M as Model participant P as Proxy handler T->>H: Return large tool output H->>S: Store original under a hash H->>M: Send compact result and retrieval marker M->>P: Call headroom_retrieve(hash) P->>S: Look up original content alt Entry is available S-->>P: Return original P-->>M: Add original as tool result M-->>P: Continue reasoning or answer else Entry expired or missing S-->>P: Retrieval miss P-->>M: Return miss information end

The sequence shows why CCR is a system property. A compressed string can be deterministic and well formed while still being unrecoverable if the store has expired, the tool was never injected, or another component produced a visually similar marker. Recovery is a contract across the compressor, state store, provider handler, and model-facing tool interface.

What the local store owns

The compression store retains the original content and metadata needed to identify how it was produced. Entries have a hash key, a lifetime, and a capacity boundary.