Inside the Filter: Turning Noise into Decision-Ready Output

RTK's filter is where a safely routed command becomes a smaller result for an AI coding agent. The key question is not how to delete the most lines, but how to match the representation to the evidence in the underlying tool's output. This chapter explains why RTK uses both specialized Rust modules and declarative TOML filters, then follows the shared runner through streaming, capture, stderr, encoding, exit status, and the guard that abandons a reduction when it would be worse than raw output.

Choose the representation that matches the evidence

Different command outputs carry meaning in different shapes. A test runner's output is a sequence of events: a session starts, tests run, failures acquire detail, and a summary closes the run. A linter's output is a collection of findings that can often be grouped by rule and file. A dependency listing has hierarchy. A Git diff has headers, hunks, changed lines, and context. A build tool mixes progress with diagnostics. Treating all of these as lines to remove would preserve neither the same evidence nor the same next decision.

RTK therefore has two broad filter systems. Compiled Rust modules handle cases where the filter must understand structure or state: JSON and NDJSON, multi-phase output, command-specific flags, injected format options, cross-command routing, or nuanced fallback behavior. A declarative TOML domain-specific language handles predictable line-oriented noise. TOML can strip progress lines, apply regular-expression replacements, keep or remove matching lines, truncate long lines, cap the number of lines, and provide a message when nothing remains. It is deliberately a line filter, not a universal reformatter.

That division is a design judgment about evidence. A line-oriented tool with stable boilerplate can be compressed by rules that say which lines are noise. A tool whose meaning depends on relationships across lines needs a parser that retains state. The difference is not the language used to implement the filter; it is the amount of interpretation required before a line can be safely omitted.

The TOML route is intentionally staged. The engine can first remove ANSI escape sequences when a tool uses terminal decoration. It can then apply chained regular-expression replacements. Whole-output match rules can short-circuit a known result with a compact message, with an optional exception that prevents the shortcut from swallowing an error. Next it can strip or keep lines, truncate individual lines, retain a head or tail, enforce an absolute line cap, and emit an on_empty message if all visible content disappeared. The order matters. A line cap applied before noise removal would spend its budget on progress text; a replacement applied after a short-circuit could never influence the result.

Built-in TOML definitions are validated during the build and embedded in the binary. The build step reads the filter files in deterministic order, combines them, checks the schema, and rejects duplicates or malformed content. At runtime, the lookup gives higher-priority on-disk filters an opportunity to match before the built-ins, while project and global custom files are trust-gated.