From Raw Output to the Right Compressor
Headroom’s central mechanism is not a single summarizer but a decision layer called ContentRouter. This chapter follows one request from incoming messages to a selected transformation, with particular attention to the choices that keep the result usable: recognizing structure, trying a reversible fold first, refusing unhelpful changes, and forwarding the original when a safe path is unavailable. By the end, you should be able to read a compression result as a sequence of decisions rather than as an unexplained token reduction.
That distinction matters because agent context is heterogeneous. A search result has repeated paths and line matches. A build log has success noise and a few failures. A diff has syntax that must remain meaningful to a patch tool. A user instruction has a different status from all of them. Applying one generic “keep the important sentences” operation to every block would ignore how the next system consumes the output.
Why routing comes before reduction
Compression is useful only if the smaller representation still supports the next action. For a log, the high-value material may be an error, stack trace, or final summary. For search output, it may be a set of relevant matches spread across several files. For structured data, it may be the relationship among fields and unusual values. For source code, even a semantically reasonable summary may be unusable if the agent needs exact lines to edit.
ContentRouter turns those differences into an explicit first question: what kind of content is this block? The current router can recognize JSON arrays, source code, search results, build or test output, unified diffs, HTML, tabular data, structured configuration, and plain text. It can also treat a block as mixed content, split it into sections, route those sections separately, and reassemble them. That lets a document containing prose and a code fence receive different treatment without requiring the caller to annotate every section manually.
Detection is not treated as infallible. The implementation combines native detection, when available, with deterministic pattern-based fallback. Cheap mixed-content heuristics can mistake a source file containing list or dictionary literals for a mixture of code and JSON. When the native detector identifies source code with high confidence, the router can trust that stronger signal instead of needlessly splitting the file. The important design point is not the particular detector; it is that classification is a mediated decision with a fallback path, not an assumption hidden inside a generic text model.
The same reasoning explains why Headroom keeps a separate provider boundary. In the current pipeline, an optional tool-result interceptor can run first, and CacheAligner can report volatile prefix content, but ContentRouter performs essentially all of the content reduction. CacheAligner does not rewrite the prompt. The router’s job is to change eligible content blocks; the caller and provider still own message assembly, authentication, and provider-specific cache semantics.
The pipeline decides in order
A useful way to understand the system is to follow the order in which a request is handled. Headroom receives a message list and model name, counts the input, and works on a copy rather than mutating the caller’s original list. It then considers the transforms enabled for that deployment. The current high-level sequence is:
- An optional canary interceptor handles narrowly defined tool-result preprocessing.
- An optional CacheAligner detects dynamic content in the prefix and emits observations.
- ContentRouter detects content shape and selects a strategy.
- The selected strategy is tested, measured, and either accepted or replaced with a safe fallback.
- The resulting messages and metrics move to the provider-facing handler.
The order creates two useful properties. First, the system can make a cheap, information-preserving change before it considers a lossy one. Second, each later decision has a chance to reject a candidate that is too large, too small to justify overhead, incompatible with the role of the message, or unsafe for a tool result that has no retrieval path.
One content block, one effective route
For a pure block, ContentRouter maps the detected type to a primary strategy. JSON arrays go to SmartCrusher. Search results, logs, diffs, HTML, tables, and configuration have structural handlers. Source code can use an AST-aware route when that feature is enabled and the protection rules allow it. Plain text reaches the project’s Kompress path or a safe fallback. If the block contains several recognizable sections, the router records a mixed decision and processes the sections independently.
“Primary” does not mean “guaranteed final compressor.” A selected handler may be disabled, unavailable, or produce no real shrink. In that case, the router can try an appropriate fallback, including the ML text path where the policy allows it. The strategy chain records that progression. This is valuable operationally: a result labeled as a fallback tells an operator something different from a direct hit, even when both messages eventually reach the model.
The router also performs work before the main strategy dispatch. It can fold repeated formatting or other recognized structure without dropping semantic records. In lossy or CCR-enabled operation, it may split search or log output by relevance, keep the higher-value records visible, and send only the lower-value tail to a more aggressive compressor. It will not apply the same follow-up to every format: for example, the implementation avoids layering a generic lossy pass over a diff because changing hunk content can undermine patch use.
Mixed-content reassembly has its own safety rules. Code fences and section boundaries are restored, and protected workflow tags are kept intact rather than passed through a compressor that might remove their contents. If a JSON array is represented as a compact string inside a mixed text block, the router unwraps that representation before reassembly so the surrounding prose does not receive an unreadable escaped value. These details serve one broader goal: compression should change the intended payload, not accidentally change the message grammar around it.
A request-time example
Imagine a tool result that contains a test summary followed by several hundred lines of output. The router first asks whether the entire block is a recognized structure or whether it contains several sections. If the output has a format-native fold, that fold is considered before a lossy model is asked to remove words. If the remaining content is clearly a build log, the log route can retain failure lines, warnings, stack traces, and the final summary while collapsing repeated success lines. If a different part is a source file or a protected instruction block, that part is handled under its own rules instead of inheriting the log decision.
The candidate then faces a second question: is it acceptable to replace the original in this message? A result that is larger than the input is rejected. A result that saves too little for the configured policy is rejected. A tool result that would become a lossy, unmarked approximation is rejected when the active contract requires CCR recovery. A cache-marked block may be rejected even when it could shrink, because the bytes have already become part of a provider cache key. These checks happen after the route has produced a candidate; classification alone does not authorize mutation.
If the candidate passes, the router stores or reuses the transformed content, records the strategy and token measurements, and places the changed content back in the same message position. If it fails, the message still moves forward with its original content. The caller does not need to understand every internal fallback to receive a valid request, but an operator can inspect the recorded strategy, timing, warnings, and route counts to learn why a block changed or did not. That separation between decision, acceptance, and observability is what keeps “automatic” from meaning “opaque.”
Why a no-op can be the correct result
Headroom treats “unchanged” as a meaningful outcome. A block below the configured size floor may not contain enough redundancy to repay detection, caching, and serialization overhead. A user or system message may be protected because it expresses the task or instructions rather than disposable observation. A cache-marked block may be left alone because altering one byte can invalidate a provider’s cached prefix. A file read may stay exact because the agent could copy its text into an edit operation.
The role of the message matters just as much as the shape of its content. Tool results are often observations that can be reduced. User and system text can be the instructions that determine how those observations should be interpreted. Recent code and code being analyzed receive additional protection. Failed tool outputs can remain verbatim when their exact error text is needed for recovery. Content that already contains a compression marker is not blindly compressed again, because a second transformation could change the bytes that a provider or retrieval path expects.
Acceptance is also tied to actual benefit. If a candidate is not smaller, or if its token measurement does not show a real saving, the router can retain the original and remember that decision in its bounded cache. A cache of skipped or accepted results avoids repeating the same work, but it is keyed by the relevant compression configuration; the same content at a different target ratio is not automatically the same result. In this design, optimization is not “always transform.” It is “transform when the current content, role, state, and policy make the change worthwhile.”
The current pipeline has a further guard for tool ground truth. A lossy strategy that produces no CCR retrieval marker is not accepted for a tool result when the active policy requires recoverability. Without that guard, the model could act on a compact-looking summary while having no way to request the rows that were removed. The result is either a recoverable compressed block, a safer structural fold, or the original content—not an unmarked approximation presented as if it were complete.
Fail-open, carefully understood
Headroom’s documentation describes transforms as failing open: when a transform fails, the request should be able to proceed with unchanged content rather than turning a compression problem into a provider outage. The pipeline also has a circuit-breaker path that can pass messages through for a cooldown period after repeated failures. Empty-output and inflation guards prevent a bad candidate from becoming an invalid or larger request.
Fail-open does not mean that every feature is silently available in every installation. The current SmartCrusher is a Rust-backed component whose native extension is required for that class. Code-aware compression, ML inference, HTML extraction, and other routes depend on optional packages or runtimes and can be gated off. The safe behavior when an optional route cannot operate is usually passthrough or another documented fallback, but the installation still determines which route exists to try.
Nor does fail-open prove semantic correctness. It protects availability and prevents obvious classes of damage: malformed JSON can pass through, an AST parse failure can return the original, and an expanding candidate can be rejected. It cannot know whether a particular retained subset answers every future question. That is why the router records strategy, timing, warnings, and route counts, and why the surrounding deployment needs representative evaluation rather than relying on a successful HTTP response as proof of quality.
Reading the decision rather than the headline
A Headroom result should therefore be read as a small decision record. Start with the input shape. Ask which route was eligible. Check whether a lossless fold or a specialized compressor ran before a more aggressive fallback. Then ask whether role, cache state, size, or retrieval requirements caused a block to remain unchanged. The token count is the outcome of those choices, not their explanation.
This view also keeps Headroom’s central theme intact. ContentRouter reduces redundant context by matching the operation to the material, while its gates protect the context that carries instructions, exact bytes, or provider-cache value. The next question is what each route actually preserves: how JSON arrays are sampled or compacted, why log and search handlers keep different signals, and why code needs a stricter boundary than routine tool output.