Scaling the Graph Across Languages and Real Repositories

CodeGraph’s scale question is not simply how many languages it names or how quickly one file can be parsed. It is whether different execution paths can preserve one trustworthy graph contract while repository size, syntax, resource limits, and agent questions change. This chapter separates the engineering mechanisms that make that possible from the benchmark evidence showing when the resulting context is actually useful.

One graph contract, different execution paths

The native Rust kernel accelerates the extraction part of CodeGraph’s pipeline. It moves language parsing and tree walking into compiled code, returns flat typed buffers across a per-file boundary, and avoids paying a language-runtime boundary cost for every syntax-tree node. That is a focused optimization: it targets the work of recognizing source structure before the graph reaches resolution and retrieval.

The kernel does not replace the rest of the system. Imports, names, inheritance, framework conventions, dynamic-dispatch synthesizers, the MCP server, the installer, and the browser viewer remain separate responsibilities. A faster parser does not by itself make a route resolve, a callback edge precise, or an explore response sufficient. Keeping the optimization at the extraction boundary helps preserve the graph contract while allowing the implementation underneath it to change.

classDiagram class SourceLanguages { parse source files } class NativeKernel { emit typed buffers } class PortableParser { recover difficult files } class GraphContract { nodes and edges } class ResolutionSynthesis { link project relationships } class RetrievalSurfaces { MCP CLI and UI } class EquivalenceGates { parity flow and regression checks } SourceLanguages --> NativeKernel : eligible files SourceLanguages --> PortableParser : errors or fallback NativeKernel --> GraphContract : per-file result PortableParser --> GraphContract : compatible result GraphContract --> ResolutionSynthesis : raw facts ResolutionSynthesis --> RetrievalSurfaces : queryable graph EquivalenceGates ..> NativeKernel : validates routing

The key property is the shared destination. Eligible files can use the native path. Files with parse errors, unsupported routing, or kernel failures can return to the portable tree-sitter/WASM path. Per-file fallback means an optimization failure costs the affected file its native speedup rather than forcing the whole project into an all-or-nothing failure. The reader is still supposed to see one graph model, not two incompatible kinds of evidence.

That promise requires equivalence gates. The current routing code keeps per-language controls, a kill switch, and a defer path for files whose recovery behavior should remain on the portable parser. The repository reports parity checks for shipped languages and uses canonical flows and agent evaluations as retrieval invariants. The relevant test is not only whether the native parser is fast. It is whether the nodes, edges, unresolved references, and downstream flows remain reliable enough for the same questions.

The fallback decision is made at the file boundary because syntax difficulty is not evenly distributed across a repository. One file may contain a dialect feature or parse error that makes native recovery unsafe while the rest of the project is ordinary.