How codebase-memory-mcp Builds a Structural Memory
deusdata/codebase-memory-mcp does not create its knowledge graph by throwing every repository file into one large parser and hoping useful relationships emerge. It builds the graph in stages, and each stage establishes facts that the next stage needs. This chapter follows that construction from discovery to publication so readers can understand two practical decisions: what evidence a chosen index mode makes available, and why a failed rebuild can leave the previous graph intact.
The ordering matters because a codebase is not a flat collection of independent documents. A call can be resolved only after definitions and imports are known. A route can be connected to a handler only after both the route-shaped operation and the handler's identity have been extracted. A semantic relation can be computed only after the graph contains the metadata that its signals use. The pipeline is therefore a sequence of increasingly specific interpretations, not a set of interchangeable scans.
Scope is a decision before parsing begins
The first question is not “What does this file mean?” It is “Which files count as the input to this index?” Codebase-memory-mcp's discovery stage walks a repository while applying several kinds of filters. It skips common version-control, editor, cache, build, dependency, and generated-output directories. It rejects binary and other non-source suffixes, applies additional filters in cheaper modes, honors Git-style ignore rules, and identifies a language for each accepted file. Optional user configuration can map an additional extension to one of the languages already understood by the executable.
This is more than housekeeping. The graph can answer only about the material that discovery presents to the rest of the pipeline. If a directory is intentionally ignored, its contents are not missing because parsing failed; they were never admitted as index input. That distinction matters when a reader interprets a result. An excluded test fixture and a failed parse have different implications, even though both can lead to fewer nodes than a person sees in the working tree.
The project exposes three principal index modes. Full mode retains the broadest discovery and runs the richest derived work. Moderate mode uses filtered discovery but still computes the similarity and semantic layers. Fast mode skips non-essential files and omits those richer similarity and semantic relations. All modes still use type-aware resolution where that resolution is available; the mode choice is primarily a choice about scope and derived evidence, not a switch between “intelligent” and “unintelligent” parsing.
That makes mode selection a question about the investigation. A quick orientation may benefit from a smaller index if the omitted files are peripheral to the question. A question about near-duplicate implementations, vocabulary that differs from identifier names, or a broad semantic relationship needs the evidence that fast mode deliberately does not create. A refresh also has to be careful not to erase a richer graph merely because a later request asks for a cheaper mode. The pipeline records coverage and can promote a requested run to the more comprehensive mode already represented by the stored generation when preserving that coverage is necessary.
The same principle explains the role of user ignore rules. They are not merely performance switches. They define the boundary of what the graph is allowed to claim about a repository. When a reader asks for exhaustive impact or absence, the right question includes whether the relevant path was inside that boundary.
Once discovery has produced its file set, codebase-memory-mcp creates a structural skeleton. The structure pass adds a project and, when Git context supplies one, a branch. It creates file nodes and the folder or package relationships that place those files within the project. This skeleton gives every later definition a stable home and supplies the qualified naming context used by searches and relationship edges.
A qualified name is important because short names collide. Two files can both define run, Config, or Handler. A node's short name is useful for human scanning, but the qualified name and repository-relative path make a follow-up query more precise. The structure pass also gives the pipeline a place to attach definitions that are not themselves files: a function belongs to a module or file, a method belongs to a type, and a route can be represented as a first-class graph object rather than as an unstructured string in a comment.
The basic construction can be read as a chain of evidence transformations:
The diagram's arrows are dependencies, not just chronology. A later stage can add evidence, but it should not silently reinterpret an earlier boundary. If discovery excluded a subtree, a later semantic pass cannot recover it by guessing. If a call cannot be resolved confidently, a later presentation layer should not turn the unresolved name into a fact merely because a query needs a target.
The graph is built in layers, then published as a generation
After the structural skeleton exists, the definitions pass reads accepted source files and calls the extraction layer. It creates nodes for functions, methods, classes, variables, modules, and other language-specific constructs, then registers callable and type-like identities. The extracted records can also carry source locations and language-specific properties when the relevant language path provides them.
The registry is the first major bridge between syntax and relationship. It gives later passes a set of known names and qualified identities against which imports and calls can be resolved. This is why definitions come before calls. Without a registry, a call pass would have only a spelling such as process() and no disciplined way to decide which project symbol, if any, that spelling denotes.
Imports then connect a file's local names to modules or definitions that the project can identify. The call pass uses those import relationships, the declaring module, and other registry strategies to resolve calls. It also preserves the difference between a proven relation and a weak match. A call edge can carry strategy and confidence information; service-shaped operations can receive a different relationship type; and an unresolved or ambiguous call can remain unresolved rather than being attached to a convenient same-name target.
That restraint is part of graph construction, not a later editorial warning. If the pipeline emitted every plausible short-name match, the graph would be dense but less trustworthy. A CALLS edge would sometimes mean “the source and target are known to be related” and sometimes mean “these names happened to match.” Downstream tracing and impact analysis could not tell the difference. Codebase-memory-mcp instead lets its resolution layer decline to create an edge when the available evidence is too weak.
The extraction stages also create relationships that are not ordinary calls. Usage and type-reference edges record how names are used without claiming that a callable invocation occurred. Test relationships connect tests to the symbols or files they exercise. Route registration can create a route node and connect a handler to it. Configuration access and infrastructure bindings can turn relationships in deployment or configuration material into graph evidence. HTTP and asynchronous operations can point toward route-like nodes so a cross-service question has something more meaningful to follow than a generic library call.
Only after the primary extraction work does the pipeline run its post-extraction passes. These add or refine evidence such as tests, Git history, configuration links, route matching, complexity, importance, semantic relations, and structural similarity. The exact set varies with mode, but the dependency is consistent: derived passes consume the graph and extraction metadata already produced. A semantic pass that uses call neighbors cannot run before calls exist. An importance score that depends on relationship structure cannot be meaningful before that structure has been accumulated.
For larger repositories, extraction can be parallelized. The workers collect per-file results, while shared graph state and the final merge are handled in controlled phases. This division is significant. Parallelism helps with the expensive independent work of reading and analyzing files, but a shared graph buffer is not made deterministic merely by adding threads. Results have to be merged in a way that preserves stable identities and does not let scheduling decide which relationships survive a cap. The project therefore separates parallel computation from serial state updates where the graph requires it.
The pipeline also treats memory as a system constraint. A richer index holds more intermediate extraction data and performs more derived passes. That can improve the questions the graph can answer, but it increases the resource cost before the result is available. The mode choice is thus a trade-off between input breadth, derived evidence, and the time and memory needed to construct the generation. The repository's measurement materials emphasize that this construction cost must be recorded separately from query latency and from an agent's eventual token or tool-call usage.
The final step is not an in-place write into the live database. Codebase-memory-mcp builds a new generation in a staging database. The staging path is unpredictable, and ownership is locked before the stage becomes visible. The pipeline dumps the graph, writes file hashes and coverage metadata, updates the full-text search structures, stores any persisted cross-file surfaces, checks integrity, and seals the staging database. Only after those operations succeed does it prepare the destination and replace the old generation.
This sequence gives failure a precise meaning. If cancellation arrives, the semantic inputs change during the run, integrity fails, or the final rename cannot be completed, the new generation is discarded or rolled back and the previous live database remains available. A failed build should not leave readers with a half-written graph that looks current. Nor should a stale stage be treated as an active writer merely because its filename exists. The pipeline uses an ownership sidecar and a narrowly recognized staging-name shape to keep live stages and dead leftovers distinct, then sweeps orphaned stages when it is safe to do so.
The same generation idea supports later query behavior. A reader can associate graph results with an opaque store generation rather than assuming that two pages fetched at different times describe exactly the same graph. If a mutation changes the materialized result, a snapshot cursor can fail instead of silently continuing over a different ordering or relationship set. Publication is therefore part of the query contract: the graph must have a coherent state before it can be a stable source of context.
When persistence is requested, or when an artifact already participates in the repository's sharing workflow, codebase-memory-mcp can export a compressed graph artifact. The export first makes a consistent SQLite snapshot with VACUUM INTO, because copying only the main file from a live WAL-backed database can miss committed frames. Best-quality export can strip secondary indexes before compression; zstd then produces the shareable file, while metadata records the project, generation-related information, sizes, and counts. The artifact is not a raw dump copied at an arbitrary instant. It is a deliberately prepared representation that can be imported and integrity-checked.
This also explains why the project treats a shared graph artifact as a cadence decision. A compressed graph is useful when teammates or repeated environments can reuse it, but rewriting it on every small change creates storage and history costs. The artifact can save reindexing work while still needing a clear relationship to the source revision and a plan for when it should be refreshed.
The construction story leads to a more useful definition of “indexed.” It does not mean that every byte of a repository has been transformed into a perfect model. It means that a selected scope passed through a sequence of extraction and derivation stages and that a validated generation was published. The resulting graph can be rich, persistent, and efficient to query while still carrying coverage records for partial parses, skipped files, and deliberate exclusions.
That is the central engineering trade-off. Codebase-memory-mcp spends computation up front to make recurring structural questions cheaper later. It gains a reusable map of names and relationships, but it must preserve the boundaries of that map: what was admitted, what was resolved, what was derived, what was omitted, and which generation a reader is seeing. Once that map has been published, the next challenge is turning it into an answer. The right query surface depends on whether the agent needs orientation, a symbol, a relationship path, exact source, or a coverage-qualified conclusion.