Running Hindsight: Storage, Workers, and the Cost of Learning

Hindsight's memory loop becomes an operational system once it must retain continuously, consolidate in the background, and answer reads under a real latency budget. This chapter connects the service boundaries and storage choices to the work they perform, then turns the write-heavy design into deployment judgment: what a small local setup can simplify, what a production setup must separate, and which cost lever matters for a given workload.

One database carries several kinds of memory state

Hindsight is PostgreSQL-first because its memory model needs several database capabilities at once. Vector search uses pgvector and approximate nearest-neighbor indexes. Keyword retrieval uses full-text indexes. Documents, memory units, tags, and operation state need ordinary relational storage and JSON fields. Graph-like relationships can be represented through links and recursive queries. Keeping those pieces in one primary system lets a memory unit, its metadata, and its retrieval relationships be updated under a coherent database boundary.

That is a deliberate trade, not an accidental dependency. The project does not hide the ordinary SQL path behind a generic storage abstraction. A generic interface could make it easier to swap databases, but it would also require another set of semantics, tests, indexes, performance characteristics, and operational instructions. Hindsight instead keeps one optimized model for the main path. The benefit is fewer moving parts for a PostgreSQL-compatible deployment; the cost is that a team unable or unwilling to operate that ecosystem has less freedom to substitute a different database.

Local development uses an embedded PostgreSQL distribution, pg0, so a developer can start with a single package and a local data directory rather than provision a separate database service. That reduces the initial operational burden, but it does not change the shape of the system: the memory state still belongs to a database, and a local embedded deployment is not automatically the right durability or concurrency choice for production.

Oracle AI Database is documented as an alternative with dedicated backend paths. The repository contains separate PostgreSQL and Oracle data-access implementations, so the alternative is more than a sentence in a deployment guide. Still, “full feature parity” is a project claim and should be read with care unless each operation and scale characteristic has been verified for the deployment. The important architectural point is that Hindsight's integrated storage model is intentional even when the backend choice varies.

The single-store choice also reduces coordination work across the learning loop. A fact, its tags, its document identity, and the links used by retrieval can be committed under one database ownership model. A consolidation worker can claim task state and write an observation against the same durable system that holds the source facts. Backups and restores have one primary target rather than separate vector, document, graph, and queue systems that can drift apart. The trade is concentration: database capacity, index health, and migration discipline become central to every operation.