How Source Becomes a Usable Graph
CodeGraph’s central engineering problem is not only how to parse code, but how to turn parsed facts into a graph an agent can query without silently confusing syntax, project relationships, and current source. This chapter follows that path from extraction to storage, resolution, and context building, then examines the scope and fallback choices that determine what the graph can honestly answer.
The pipeline starts with syntax, not meaning
The first stage is extraction. CodeGraph uses tree-sitter-based parsing to inspect source files and emit nodes and raw edges: the functions, classes, methods, imports, declarations, and explicit relationships that can be recognized from the syntax. At this point the system has identified structures inside files. It has not necessarily established which name a call refers to across the project.
That distinction is easy to miss because a parsed call can look like a complete relationship. Consider a method that invokes save(). The extractor can recognize that there is a call expression and record its location. It may not yet know whether save() is a method on the current class, an imported function, a method supplied by an inherited type, or one of several same-named definitions elsewhere. Extraction answers, “What does the source appear to contain?” Resolution answers, “Which project object does that reference connect to?”
Keeping those stages separate makes the system easier to reason about. Extraction can concentrate on language syntax and source ranges. Resolution can use project-wide information, imports, names, inheritance, and framework patterns. The graph can retain unresolved references between the stages instead of forcing an early guess that later work cannot undo cleanly.
The boundary also explains why support for a language is more than recognizing its file extension. A useful graph needs nodes with meaningful ranges and relationships with the right receivers, declarations, and names. A parser may recognize a method but fail to capture its body correctly, or recognize a route declaration without knowing which handler it names. CodeGraph’s later stages can only resolve what extraction has represented accurately enough to be queried.
Extraction is therefore a contract about shape and location. The parser must preserve where a node begins and ends, which file and language produced it, and which source line contains a reference. Those coordinates are later used to show a caller at its call site, to open a flow at the line that makes the next hop, and to replace or remove the right records during synchronization. A syntactically recognized symbol with a wrong range can be as damaging to retrieval as a symbol that was never recognized: the agent may be sent to the wrong body or the index may retain stale relationships after an edit.
The extraction stage also has to tolerate imperfect source. Real repositories contain generated files, dialects, partial configurations, and files with syntax errors. A robust index can record an error or defer a file without pretending that its graph is complete. That choice preserves the boundary between “the source could not be fully extracted” and “the project contains no such symbol,” which will matter when an agent interprets a sparse result.
Persist before you retrieve
After extraction, CodeGraph stores the graph in a project-local SQLite database. The database holds nodes, edges, tracked-file records, and unresolved references, with full-text search over names, qualified names, docstrings, and signatures. This persistence changes the cost model of an agent query. The system does not need to reparse the repository and rediscover every relationship for each prompt; it can search the precomputed structure and traverse the relevant edges.
SQLite is important here less as a brand choice than as a boundary of ownership. The project keeps its index beside the project rather than requiring a remote graph service. The stored database belongs to one codebase and can be opened by the CLI, MCP server, browser viewer, or library API. Those interfaces may ask different questions, but they draw from the same persisted representation.
The stored file record is part of the pipeline, not bookkeeping added afterward. It can retain a content hash, modification time, language, size, indexing time, node count, parse errors, and generated-file status. Those facts let later synchronization compare the source on disk with the version represented in the database. They also give retrieval a basis for distinguishing a generated or mechanically dense file from the hand-written code that explains a domain behavior.
Persistence makes incremental work possible as well. A full index establishes the project-wide baseline. A later sync can inspect which files were added, modified, or removed, replace the affected nodes and edges, and update the relationships that depend on changed definitions. Search and impact queries can then reuse the existing graph rather than waiting for a full rebuild after every edit.
The difficult part of an incremental update is that a definition can affect files that were not edited. If a new or changed symbol competes with an existing same-named definition, a previously resolved reference may need to be reconsidered. The stored graph therefore supports more than a changed-file list; it must retain enough project-wide information to keep resolution convergent with a clean rebuild. A body-only edit may alter source bytes without changing the set of definitions, while adding, removing, or moving a definition can change the meaning of edges elsewhere.
This is why file metadata and unresolved-reference records belong in the same story as the node and edge tables. The database is not only a place to save answers. It is the state from which the next synchronization decides what changed, what must be removed, what can be reused, and what needs another resolution pass. If that state is incomplete, later retrieval may be fast but systematically wrong.
That reuse is the practical reason to separate graph construction from question answering. The expensive work of parsing and resolving is amortized over many questions: architecture questions, caller lookups, impact checks, browser navigation, and context requests. The graph is not merely a response cache. It is a durable intermediate representation from which several kinds of evidence can be derived.
Resolution turns names into project relationships
Resolution is the stage that gives the raw graph its project-level meaning. CodeGraph resolves imports to the files they point at, calls to definitions where the evidence supports a match, inheritance relationships, and framework-specific patterns. It can also run selected synthesizers for relationships that are not expressed as direct calls. The result is a graph in which a query can move from a symbol to its callers, callees, dependents, routes, or implementations.
Name matching alone is not enough. A large repository may contain several methods with the same short name. Imports and qualified names can narrow the candidates. The receiver type, enclosing class, language rules, and framework conventions can provide more context. When that context is missing or ambiguous, the resolver may keep the reference unresolved rather than produce a confident but wrong edge. The graph’s quality therefore depends not only on how many references it links, but on whether the links preserve the distinctions the source makes.
Framework resolution is a useful example of why the stage exists. A web route may declare a path and a handler through a framework-specific decorator, configuration file, or registration call. A direct syntax walker can see the route declaration and the handler definition separately. Resolution adds the relationship that makes the application flow visible: external route to handler, then handler to the rest of the call graph. The same principle applies to selected client routers, ORM patterns, and cross-language bridges.
Resolution is also where the system must decide how to handle indirect behavior. Some patterns have enough structure to synthesize a bounded edge, such as a callback registration paired with a dispatcher. Others remain dynamic and should be surfaced as a boundary. The pipeline does not become more correct by forcing every unresolved reference through a generic name match. Its job is to convert evidence into relationships while keeping the evidence strength attached.
Once resolution has produced a usable graph, query and context layers can traverse callers, callees, impact radius, and related symbols. They can format the result as Markdown or JSON for an agent, or as structured views for the browser. This is the point at which the graph becomes surgical context: not because the source has disappeared, but because a question can select a meaningful path through it.
The order matters. A fallback parser still has to produce graph-compatible facts. An unresolved reference still needs to remain visible if the query is to explain where the path stops. A context builder cannot recover a symbol that extraction never emitted, and a flow query cannot prove runtime behavior merely because resolution found a plausible static path. Each stage narrows uncertainty; none erases it completely.
Scope is a correctness choice
A repository contains much more than the source an agent should usually reason about. Dependencies, build products, caches, generated bundles, ignored files, and very large blobs can dominate an index while contributing little to an architecture answer. CodeGraph excludes common dependency, build, and cache directories by default, honors .gitignore, and skips files larger than 1 MB. Optional configuration can add custom extensions, include first-party source that a second version-control system hides from Git, exclude tracked directories, or deprioritize noisy paths.
These rules are not only performance optimizations. They define the boundary of the graph. If a generated bundle enters the index, it can create name collisions and consume retrieval capacity. If genuine source uses a nonstandard extension and is skipped, a real part of the project becomes invisible. If a committed vendored directory cannot be excluded, impact and ranking can be distorted. Scope decisions therefore affect both speed and the meaning of a query result.
The right response is not to index everything by default. It is to make the scope legible and adjustable. A user can opt a directory back in when it contains relevant vendored code, map a custom extension to a supported language, or exclude a tracked tree that is not part of the product’s logic. The graph then reflects a declared interpretation of the project rather than an accidental walk of every readable file.
The pipeline also has to manage resource limits. Parsing is I/O- and CPU-intensive, so CodeGraph uses worker pools, batches file work, bounds parse time, and recycles workers when the underlying runtime cannot readily reclaim parser memory. Resolution has its own costs and can use parallel work when the project is large enough for that to pay off. These choices affect completion time and memory pressure, but they are subordinate to the reader-facing contract: a faster partial graph is not a successful index if files silently disappear.
Parallelism introduces its own ownership problem. A worker can parse a file, but the graph still needs a consistent write boundary and a predictable resolution phase. A timed-out parser must not leave half a file’s nodes in place. A sync must not erase a pending edit that arrived while another sync was running. The precise implementation handles these concerns through orchestration, locks, and result accounting; the reader-facing consequence is simpler: an index should either report a known limitation or leave a coherent record that later work can repair.
That is why the implementation retains errors and counts discovered, skipped, indexed, and changed files. An index can be operationally fast yet incomplete. Explicit status gives the user a chance to notice an unsupported extension, a parse problem, or a failed worker instead of treating an empty result as proof that the code does not exist.
Portable fallback is another part of that contract. The current extraction path can route eligible languages through a native Rust kernel and return to the portable tree-sitter/WASM path for files that do not qualify, contain parse errors, or trigger a kernel failure. The optimization changes how extraction is performed, not what a reader is supposed to believe about the resulting graph. Per-file fallback is valuable precisely because it prevents one difficult file from turning an entire project into an all-or-nothing decision.
Unresolved references serve a similar purpose. They preserve the observation that a reference existed and allow later resolution or synchronization to revisit it. A failed match is therefore not silently discarded, and a query can distinguish “the code contains this reference, but the current graph cannot bind it” from “the source contains no such reference.” This is a recoverability feature as much as an uncertainty feature.
The pipeline stops before the agent receives a final answer. It does not execute the program, observe a database, run a test, or validate that a route is reachable in production. It produces a structured, current-as-possible model from static evidence. Query and context building then choose a bounded presentation of that model. The next problem is consequently not how to build the graph, but how to expose it to an agent without returning either a noisy dump or a misleadingly thin slice.