What Context Mode Can—and Cannot—Save

mksglu/context-mode addresses the opening problem by changing where computation, evidence, and session state live before they reach the model’s active context. The final question is how far that solution really goes: what do its statistics measure, which workloads benefit, what costs remain, and what must an operator verify on the actual host? A defensible judgment treats context-mode as a boundary architecture with conditional gains—not as a universal compression switch or a substitute for host security and evidence review.

First, measure the right bytes

The phrase “context savings” can hide several different quantities. Context-mode’s tools may read a large file inside a subprocess, index a long document in SQLite, return a small derived answer, return an exact search section, or record an event about what happened. Those bytes have different relationships to the model’s context window. Combining them into one number without a scope produces a precise-looking but ambiguous result.

The project’s statistics distinguish at least five useful categories:

• Bytes avoided are raw bytes that context-mode kept from entering the model context by processing, indexing, caching, or redirecting them.
• Bytes returned are the results the model actually received from context-mode tools, including the selected output needed to continue reasoning.
• Bytes retrieved describe the content the model accessed from an indexed store. They are the cost of looking back into material that was previously kept out, not a second measure of the original raw payload.
• Event-data bytes belong to the local session and analytics infrastructure. They describe what hooks recorded, not what the model saw in the conversation.
• Lifetime totals aggregate storage and session activity across a broader scope than one current conversation.

The accepted strict-compression formula deliberately keeps these meanings separate. For a non-empty conversation, the “without” quantity is bytesAvoided + bytesReturned. The “with” quantity is max(1, bytesReturned). The displayed percentage is the share kept out of the model-context transfer under that definition. When both quantities are zero, the report uses an honest empty-state message rather than drawing a meaningless 0% or 100% bar.

The exclusion of event-data bytes is not a minor accounting detail. Hooks may capture repeated rule content, file events, decisions, and other payloads so that the system can restore or analyze a session. Those records can be large, but they are not automatically part of the conversation the model had to carry. Adding them to both sides of the per-conversation bar makes the infrastructure look like context consumption and can sharply understate the effect of a redirect. The project retains those bytes for capture counts and broader lifetime views, where they answer a different question.

Retrieval also needs care. A sandboxed computation that reads a 500-row file and prints a count produces work output. A ctx_search response gives the model an exact section it intentionally requested. The latter is a context cost, but it is a controlled cost: the model pays for the evidence it needs instead of receiving the entire source preemptively. Context-mode’s own accounting records this distinction so “kept out” does not mean “never available.”

This gives ctx_stats a useful role, but not an all-purpose scorecard. Its report combines live counters, persistent session information, content-store state, conversation/window measurements, and lifetime aggregates. Each section should be read according to its denominator. A per-conversation transfer bar can tell an operator how much raw material was kept out of the current window. It cannot by itself say whether the agent reached the right conclusion, whether the computation was fast, or whether the retrieved passage was semantically sufficient.

The benchmark is evidence, not a universal constant

The project’s benchmark document reports 21 scenarios using real captured fixtures, 376 KB of raw data, 16.5 KB of context consumed, and 96% overall context savings. It also reports that all 125 tests in the listed suites passed at the time of the benchmark. The README uses a separate 98% product headline, reports examples such as a Playwright snapshot falling from about 56.2 KB to 299 bytes, and describes a full-session illustration in which 315 KB becomes 5.4 KB.

These numbers can coexist because they answer different questions. The benchmark’s aggregate covers a defined fixture set and tool mix. The README’s headline is product positioning, and its scenario values describe particular workflows. A summary produced by ctx_execute_file can be far smaller than an exact chunk returned by ctx_search. A cache hit can avoid a network fetch and re-indexing, while a fresh exact lookup intentionally returns evidence. The denominator, workload, and output contract determine the result.

The benchmark document itself explains why exact retrieval has lower apparent savings than compressed execution. For logs, CSV data, test output, and build output, a short aggregate may be the useful answer. For documentation, API references, and code examples, the exact text is the useful answer.