Why the Wiki Lives in Markdown
For ai-memory, storage is not a neutral implementation choice. The project must preserve a readable account of what coding agents learned while still answering searches quickly and keeping hook writes safe. This chapter explains why Markdown and git sit on the authoritative side of that boundary, why SQLite remains essential but derived, and how versions and typed project identity keep a shared memory from collapsing into a pile of colliding filenames.
Two layers, one source of truth
The durable center of ai-memory is a wiki made of ordinary Markdown pages. Those pages are stored in a git-backed tree, so a person can inspect them with familiar tools, open them in a Markdown application, compare revisions, or synchronize the directory independently of the agent service. A decision, gotcha, procedure, or session account is therefore not trapped inside a database record whose meaning can be recovered only through one API.
SQLite does a different job. It maintains the indexes and operational records that make the wiki useful at runtime: full-text search, entities, links, sessions, observations, handoffs, audit entries, and optional embeddings. It also tracks page versions and the metadata needed to decide what is latest, what is superseded, and which project a row belongs to. The database is the system's working map of the wiki, not the only copy of the knowledge the map describes.
That distinction matters when a human edits a page or when the service needs to repair itself. The watcher can notice changes made outside the normal write path and reconcile them. A deliberate reindex can rebuild page, link, and full-text structures from the Markdown tree. The project can also checkpoint the wiki in git before operations that change it and restore an individual page from an earlier checkpoint. In each case, the file is treated as the thing to understand, while the index is reconstructed to agree with it.
This is not a claim that the two layers form one perfect transaction. A filesystem rename and a SQLite commit cannot be made into a single universal transaction merely by placing them next to each other. ai-memory routes page mutations through shared wiki/store operations, writes files atomically, updates indexes in the same database transaction as the relevant rows, and uses reconciliation to repair crash windows. But the recovery model remains explicit: rebuilding from the wiki can restore wiki-owned pages and links, not every operational row the service has accumulated.
The asymmetry is important. A clean Markdown tree can rebuild the project and page index, but it cannot reconstruct all sessions and raw observations, open or accepted handoffs, users and credentials, audit history, access counters, or embedding vectors that existed only in SQLite. Those require a database backup. Conversely, a database snapshot without the wiki does not preserve the human-readable source that the system says is authoritative. The practical lesson is simple: back up both layers, and do not confuse “the index can be rebuilt” with “all state has a second copy.”
Imagine a page write succeeds on disk and the process stops before every surrounding operation has settled. The file can still be inspected and a later reconciliation can rebuild its page and link indexes. But a handoff or observation that existed only in the database may need the online backup or a database restore to return. This is a healthier failure story than silently treating the two layers as identical: the operator knows which artifact can repair which class of loss, and the system does not promise recovery it cannot provide.
Why one SQLite writer protects the boundary
The database's derived status does not make consistency optional. If a page were written to Markdown and returned to the caller while its search row appeared later—or failed to appear at all—the next agent could receive a result that disagreed with the file a human would open. ai-memory therefore commits the data and its indexes together inside the SQLite transaction that owns the operation. A query should not have to guess whether a successful write is still waiting for a detached indexing task.
All database writes pass through a single writer actor. Reads can use a read-only pool, but one dedicated writer owns the write connection and serializes mutations. This is a deliberate answer to SQLite's concurrency model and to the service's hook traffic. The writer gives page writes, observation ingestion, handoff changes, audit records, and index updates one ordered path. It also creates a visible point at which backpressure can be applied.
The trade-off is not that one writer makes the system infinitely scalable. It makes the consistency boundary legible. Hook clients can remain fire-and-forget and return quickly; the service can accept work into a bounded queue or refuse it when saturated rather than creating an unbounded backlog. The expected project corpus is measured in hundreds to low thousands of pages, not an internet-scale document warehouse. For that shape, an embedded database with a controlled writer can be easier to inspect and recover than a distributed search and graph stack whose synchronization failures are harder to see.
The same principle explains why vectors remain ordinary derived rows rather than a separate mandatory vector database. A vector index can be rebuilt or refreshed without becoming the authority for page content. If a provider, model, or dimension changes, stale vectors can be recognized and ignored until re-embedding completes. The retrieval layer may use a vector as one signal, but the Markdown page and its canonical metadata remain outside that signal's control.
A page changes by supersession, not silent replacement
Suppose a project first records one database choice and later migrates to another. A naive note system might edit the original text in place. ai-memory instead keeps a page-version chain: a newer version supersedes an older one, and ordinary present-tense retrieval uses the latest version while history remains available. The old page is not necessarily the answer to what the project believes now, but it can explain what the team believed when an earlier decision was made.
Supersession is not a merge algorithm. If two operators write competing versions of the same page, the later version becomes latest and the earlier version remains reachable in the chain. The system preserves history rather than claiming that it reconciled the disagreement. A human or a later lint process still has to decide whether one account corrects the other, whether both describe different conditions, or whether the page should be rewritten with an explicit relationship.
This model also makes a page's lifetime different from a session's lifetime. A session page may be a chronological account of one run. A durable decision page can be revised as the project changes. A page's path can stay stable while its content and evidence evolve. Retrieval can prefer the current version without erasing the evidence that a later temporal query or audit may need.
The wiki's documented Open Knowledge Format conformance extends the same idea beyond ai-memory. A project directory can be treated as a portable bundle of Markdown concept files with standardized type, provenance, and freshness fields. The receiving tool does not need to understand ai-memory's whole server to read the decisions and procedures. An explicit OKF export exists, but it is an export of files the wiki already uses rather than a second secret truth assembled only for export.
That portability has a boundary. OKF describes the knowledge bundle, not the complete SQLite operational history or any agent's private native session database. It helps another person or tool read the project knowledge; it does not reproduce handoff ownership, audit rows, embeddings, or native resume state. Portability is strongest when the artifact being moved is clearly identified as knowledge rather than confused with all the machinery that produced it.
Identity is part of storage, not a naming afterthought
A file path alone is not enough to identify a memory page. Two projects can both have a page called “the first database decision,” and two checkouts can have the same basename while belonging to different clients or worktrees. ai-memory carries a typed coordinate—workspace, project, and page path—through its domain rows. Human-readable names remain useful, but stable identifiers behind them keep a rename from becoming a filesystem collision.
The on-disk layout follows that rule. Project directories are keyed by stable project identity, while the human project name is metadata that can change. Renaming a project can update the name without moving every page file. A project can therefore be renamed after a checkout moves without treating the new name as a new knowledge base. Destructive operations can target the intended workspace and project rather than deleting every matching path in a flat tree.
The marker file makes the coordinate explicit at the edge of the system. A repository can declare its workspace and project, or ask ai-memory to derive the project from the main repository root so subdirectories and worktrees do not silently create separate buckets. Static MCP clients can send the same explicit scope on project-level calls. The nearest marker wins, and the service can report which scope was resolved, which makes routing errors diagnosable rather than mysterious.
This matters most in a shared server. A current-project pointer that is global to the process could let two agents running at once overwrite each other's implicit destination. The current design can isolate that pointer by actor or session, while explicit workspace/project arguments remain the safe option for clients that cannot forward a real lifecycle session identity. Scope prevents one class of collision; it does not create page-level permissions. Once a page belongs to a shared project, authenticated project operators can generally read it.
The same distinction appears in repository-native decisions. If a project already maintains architectural records in its own repository, that repository may be the authority. ai-memory advises excluding those paths from capture so a consolidation pass does not create a stale wiki copy that later ranks as if it were current. If no repository-side record exists, a structured wiki decision can provide the durable account. The important choice is not which directory wins by convention, but which artifact owns the fact and how a reader can tell.
The trade-off behind the architecture
Putting Markdown first gives ai-memory human visibility, ordinary version control, and a portable knowledge boundary. Putting SQLite beside it gives the service transactional writes, fast search, entity and link indexes, session state, and operational audit. A single writer and same-transaction indexes keep the derived layer coherent enough for live use; reindex and backups make failures recoverable within stated limits.
The cost is that operators must understand two kinds of state. The wiki is the durable semantic surface, while the database also contains valuable operational history that cannot be regenerated from pages. A distributed database might offer a different scaling story, but it would not automatically offer a clearer source of truth. ai-memory chooses the arrangement because the central continuity problem is not only finding a record quickly. It is allowing people and agents to inspect what the record means, revise it when circumstances change, and carry it into another tool without surrendering ownership of the underlying knowledge.
With that ownership model in place, the next question is no longer simply where memory is stored. It is what a later agent should classify as raw evidence, compiled project knowledge, or current authority—and how retrieval can connect those surfaces without flattening them into one kind of truth.