A Memory You Can Inspect
OpenHuman’s memory system answers a narrower and more demanding question than “can an AI search my documents?” It asks how scattered messages, notes, and connected data can become durable context that the agent can use and the person can examine. This chapter follows that path—from source item to retrieved evidence—and explains why the project treats memory as a pipeline and a user-owned Markdown artifact rather than as an invisible similarity index.
Why ingest is a pipeline
A connected inbox is not yet useful memory. It contains duplicates, short acknowledgements, signatures, automated notices, inconsistent names, changing threads, and material with very different importance. If every item is embedded and presented to the model in the same way, the system has not solved context; it has only moved noise into a different database.
OpenHuman’s documented Memory Tree therefore begins before retrieval. A source adapter brings in a message, document, conversation, local note, or integration result. The system canonicalizes it into a normalized Markdown-like representation with provenance, then divides it into bounded chunks with stable identities. That normalization is important because later stages need to compare, score, summarize, and cite material without losing where it came from.
The next decision is admission. The documented scoring layer combines signals such as content length, lexical diversity, source and metadata weight, interaction, and entity density. Obvious high-signal and obvious low-signal material can be handled with cheap signals. Borderline chunks may receive deeper model-based importance or entity analysis. The point is not to claim that a score understands meaning perfectly. It is to keep the tree dense enough that summaries remain useful and to avoid spending an expensive model call on every trivial line.
This is also a form of information triage. A direct message containing a decision, a deadline, or several named entities is more likely to deserve promotion than an automated footer or a one-word acknowledgement. The system can use interaction and source metadata as evidence that something mattered, while entity density and later enrichment help distinguish a substantive item from a long but empty notification. That does not turn the score into a universal measure of importance. It makes the admission decision explicit enough to inspect and revise as the system evolves.
A dropped chunk is not necessarily erased. The scoring documentation says its provenance row can remain for diagnostics even when the chunk is not admitted into a summary buffer. That is a small but consequential design choice. It separates “the system chose not to promote this item into active memory” from “the item never existed.” A user investigating an omission can therefore ask a different question: was the source unavailable, was the item rejected as noise, or did retrieval simply fail to surface it?
After admission, heavier work can proceed in the background. Entity extraction enriches a chunk with people, organizations, topics, handles, or other stable references. Embeddings support semantic recall. Sealed groups of material are summarized into higher-level nodes, so the system can represent a long period or a large source without placing every raw item into every prompt. The interface does not have to wait for the entire enrichment path before the initial write is acknowledged.
This separation of hot and cold paths is what makes a memory system usable at personal-data scale. The hot path records and queues work; the background path computes richer signals and summaries. That division introduces eventual consistency—new material may be stored before its entities, embeddings, or higher-level summaries are ready—but it avoids making every ordinary sync behave like a synchronous research task.
From a message to a memory artifact
The most visible expression of this pipeline is the Obsidian-compatible vault. OpenHuman’s documentation says that the same chunks used by the agent are written as ordinary Markdown files that a person can open, browse, edit, and link. The vault is not only an export format added after the “real” memory is built. It is presented as part of the memory relationship: if the agent’s recollection cannot be read, it is difficult to trust or correct.
That changes the meaning of an error. Suppose OpenHuman associates a planning note with the wrong person because two names are similar. In an opaque vector store, the user may only observe a strange answer. In a readable vault, the user can inspect the generated note, check its provenance, edit a supporting Markdown file, and understand what future retrieval may see. This does not guarantee that the next answer will be correct. Retrieval, summarization, and model interpretation can still fail. But the failure has a surface where a person can intervene.
The same property supports portability. The documented vault uses ordinary Markdown and standard wiki-link conventions. A user can open it in Obsidian or another editor, maintain hand-written notes, and ingest those notes alongside connected sources. The system is therefore not claiming that the assistant owns the user’s memory. It is trying to make the assistant a processor and navigator of artifacts the user can retain independently.
There is a practical trade-off. Readability exposes state, but it also exposes the system’s compression choices. A summary is not the same thing as the full source. A concise note can be easier to navigate while omitting a qualification that matters to a later decision. That is why provenance and drill-down are as important as the summary itself. A memory system needs a compact view for orientation and a path back to the underlying material when precision matters.
OpenHuman’s integration design extends the same pipeline over time. A connected service can be exposed as an agent tool, used as a memory source, contribute evidence to personalization, and emit live triggers. The documented auto-fetch loop checks active connections on a periodic cadence, maintains per-connection sync state and cursors, applies budgets and provider-specific intervals, and records failures without allowing one broken connector to stop the entire scheduler. This makes the memory tree a changing context layer rather than a one-time import.
But “continuously refreshed” does not mean “always current.” A provider may sync only at its configured interval. An OAuth scope may exclude material. A cursor can lag, an API can fail, and a source can be exposed as a tool without having a native ingest path. The memory pipeline inherits all of those conditions. Freshness is therefore a property of the source and sync contract, not an automatic consequence of connecting an account.
The loop in the diagram has two directions. The forward path compresses a messy stream into navigable context. The reverse path is the accountability path: retrieval can lead back to a raw leaf, and a user correction can become part of what the system will ingest and retrieve later. Without that reverse direction, compression would be convenient but difficult to audit.
Retrieval means navigating evidence
OpenHuman’s documented retrieval surface is broader than “find the nearest vectors.” A query can be scoped to a source, narrowed to a time window, routed through entities, expanded from a high-level summary, or resolved to raw leaves for exact text. These operations answer different questions.
A similarity search is useful when the user remembers an idea but not its wording. A source-scoped query is better when the user asks what happened in a particular channel or document. A time-window operation matters for questions such as “what changed since yesterday?” An entity lookup matters when “Alex,” a project codename, and an email address need to resolve to one stable identity before the system combines evidence. A drill-down matters when a high-level summary identifies the relevant period but the answer requires the underlying message.
Imagine asking what changed in a project while you were offline. A useful retrieval path might first constrain the time window, then identify the project and people involved, then search the relevant source summaries. If a summary says that a decision changed, the agent can drill into the summary’s children and fetch the underlying leaves before presenting the claim. If the question instead asks for a broad orientation, the summary may be enough. The important point is that the retrieval operation follows the question: breadth, scope, identity, recency, and exact citation are not interchangeable requirements.
This is why the project describes memory as navigation as well as recall. A summary node can provide orientation; its children provide detail; a leaf can provide the source text and provenance needed for a citation. The agent does not need to inject the entire history into every turn. It can start with a compact representation and fetch more detail when the question demands it.
The entity layer also creates a bridge between memory and relationships. The documented system can maintain canonical identifiers and derive co-occurrence links between entities mentioned together. A question about a person, project, repository, and event can therefore be routed through both semantic similarity and the structure of who or what appears together. A graph in this context is not a replacement for the underlying documents. It is a map for finding the documents that may matter.
Source scoping adds a privacy and relevance boundary. An agent profile can be restricted to a whitelist of sources so that a work-focused assistant does not casually retrieve a private personal inbox. The design distinguishes tagged source material from the agent’s own working conversation context, allowing the assistant to retain the immediate task even when connected-source access is narrowed. That is more than a ranking preference. It is an explicit decision about which parts of a person’s world a particular agent is allowed to recall.
Still, structured retrieval should not be confused with guaranteed truth. Canonical identity can be wrong. Entity extraction can miss a reference. A summary can flatten a disagreement. A source scope can be correctly enforced while the model misinterprets what it retrieves. The value of the structure is that it makes the chain of interpretation easier to inspect: which source, which time range, which entity, which summary, and which raw leaf contributed to the answer?
The limits of the tree
The repository’s documentation is not fully aligned about the exact current shape of the Memory Tree. Some user-facing pages describe source, topic, and global trees as separate projections. A current developer description says the historical global and topic projections were removed in favor of walking source trees together with an entity index. The safe conclusion is not that one page can be ignored. It is that the stable design claim is narrower: OpenHuman maintains hierarchical, source-aware, provenance-bearing memory and offers structured retrieval over it, while the precise projection model is version-sensitive.
The same caution applies to ownership. The current memory host documentation says the storage, ingestion, and tree mechanics are provided through an extracted memory stack, while OpenHuman retains the RPC surface, agent tools, guards, source scoping, and host integration. For a reader, this implementation migration matters less than the interface it preserves: the assistant still depends on a memory provider, and the host still owns decisions about what the agent may access and how memory is exposed.
The deeper lesson is that persistence and understanding are different achievements. OpenHuman can make a source durable, score it, summarize it, and return a provenance-bearing hit. It cannot guarantee that every important fact will be admitted, that every summary will preserve every qualification, or that a model will reason correctly over the retrieved material. Its strongest contribution is to make context less ephemeral and less opaque. Its remaining responsibility is to show where that context came from and where it may be incomplete.
That distinction sets up the next problem. Once OpenHuman can supply a rich, inspectable context, the agent still has to do something with it. The next chapter examines the execution layer: how a context-rich turn becomes a bounded sequence of tool calls and delegated work that can pause, recover, and report failure rather than disappearing into an opaque loop.