How Facts Become a Connected Memory
Now that Hindsight has turned conversation into durable facts, the next question is why those facts can behave like memory rather than a pile of extracted sentences. This chapter examines the connections around a memory unit: entities, semantic similarity, time, causes, attachments, tags, and bank scope. The practical payoff is a way to follow a question such as “Why did this happen last spring?” while also seeing where Hindsight deliberately stops connecting information and where an inferred relationship remains only a retrieval aid.
A name is a route through memory
A sentence can mention a person without explaining how that person relates to other facts. If one conversation calls someone “Alice,” another says “Alice Chen,” and a third uses “Alice C.,” a text-only system may treat them as separate search terms. Hindsight tries to resolve such variants into a shared entity. It uses name similarity, co-occurrence, and temporal context so that a query about Alice can reach facts stored under related surface forms.
Entity resolution creates a route through the bank. Suppose the retained facts say that Alice leads Project Atlas, Project Atlas uses Kubernetes, and the Kubernetes cluster suffered an outage. A question about whether Alice was affected by a recent issue may not resemble the wording of all three facts. The entity path—Alice to Project Atlas to Kubernetes to the outage—can still provide a useful bridge. The graph is doing something different from semantic similarity: it is following a shared object or relationship rather than asking whether two sentences sound alike.
That route is valuable precisely because it can recover indirect context. A project decision may be stored in one document, a deployment event in another, and the responsible team in a third. Linking their entities lets the system assemble a larger picture without requiring every source to repeat every name. It also means that entity quality becomes part of retrieval quality. A missing link can hide relevant context; a mistaken merge can attach one person's facts to another person's history.
Hindsight's own documentation treats resolution as a judgment call, not an identity oracle. Close variants can merge automatically. Dissimilar nicknames and formal names may not be unified by name alone, even when a human would know they refer to one person. In a dense bank, a short new name that resembles an existing entity and appears beside the same related entities can sometimes be absorbed into the existing entity. The lesson is architectural: an entity edge helps the system navigate evidence, but it does not prove that the underlying identity decision is correct.
Semantic links provide another kind of connection. Two facts can be related in meaning even when they do not share an entity or exact wording. A statement about repeated timeout errors may be close in meaning to a statement about a service becoming unreliable under load. These links give retrieval a way to move through topics, while entity links give it a way to move through objects. Hindsight prepares both kinds of relationship during or after retain so that later search is not forced to rediscover every connection from scratch.
The four relational signals can be separated by asking what kind of uncertainty a question contains. If the uncertainty is “which Alice does this refer to?”, entity resolution matters first. If it is “what else happened around that date?”, temporal links provide a neighborhood. If it is “what explains the failure?”, causal links are more useful than mere closeness in time. If it is “what was the team discussing under different wording?”, semantic links can bridge the language. A single question can need several signals, but they should not be interpreted as interchangeable evidence.
This distinction also explains why a graph is not a replacement for the source text. A link says that two memory units are worth considering together under a particular relationship. It does not erase the wording, timestamp, speaker, or context from which the relationship was formed. Hindsight can use the link to retrieve an indirect fact, then let a later operation inspect the underlying units before making a conclusion. The graph increases the reachable evidence set; it does not turn reachability into truth.
The local pieces fit together into a larger structure:
This is not a promise that every internal table or edge is exposed to an application in exactly this form. It is an operating model: a bank scopes memory, documents provide source material, memory units carry facts and metadata, entities and links connect units, and later observations cite the units that support them. A fact is therefore more than its text. It is text situated in a network with boundaries.
Time and causality answer different questions
Time is not one generic relevance signal. Hindsight records when an event occurred and when the system learned about it. Those dates answer different questions. If someone tells the agent in January 2025 that Alice got married in June 2024, the event belongs in a query about what happened in 2024. The January mention may still affect freshness or explain when the bank acquired the information. A system that keeps only the ingest date loses the historical event; a system that keeps only the event date cannot tell when the information became available.
This distinction also changes how the system handles relative language. “Last spring” is meaningful only relative to a reference date. Hindsight's extraction rules resolve relative expressions against the event date supplied with the content when possible. When a source names only a year or month, the representation preserves that uncertainty as a period rather than inventing a precise day. That protects later time-aware reasoning from false precision. A memory dated “sometime in 2024” should not silently become a claim about January 1.
Temporal links add context around an event. Facts close in time can be connected so that a query about what else happened around a milestone can reach nearby events. The link does not say that proximity caused the events or that every nearby fact is relevant. It says that time is a useful navigation signal. A deployment change, an alert, and a rollback may be worth considering together because they happened close together, even if their texts use different vocabulary.
Causal links answer a different question. If one fact says that a team worked long hours and another says the team later experienced burnout, a causal edge can represent the stated relationship between them. A query asking why something happened can follow that path. The causal relation is stronger in meaning than simple temporal proximity, but it is still grounded in an extracted or supplied relationship. A fact that happened earlier is not automatically a cause. Treating every before-and-after pair as causal would turn chronology into a story the source never told.
These distinctions make the same cluster of facts useful for several questions. “What happened around the outage?” calls for temporal context. “What caused the outage?” calls for causal evidence. “What was Alice's team working on?” may depend first on entities and then on semantic links. “What did the team know at the time?” requires attention to when facts were learned, not only when events occurred. Hindsight's connected representation does not answer all four questions with one score; it gives later retrieval several signals to combine.
The connections also have different failure modes. Entity resolution can merge close names incorrectly. Temporal extraction can misread a relative date if the event reference is absent or ambiguous. A causal relation can be omitted, overstated, or attached to the wrong earlier fact. Semantic links can connect concepts that are similar but not relevant to the user's specific question. These are reasons to preserve source text and metadata alongside the links. A graph edge should help a reader find evidence, not replace the evidence.
Attachments and tags define where connection should stop
A screenshot or chart can contain the decisive detail in a document. Hindsight's multimodal retain path treats an image or file as an ordered block beside the text that refers to it. The extractor can then associate a fact with the attachment it actually needed. If a screenshot contains a request ID that is absent from the surrounding prose, the resulting fact can carry that screenshot as provenance. If the fact came entirely from the prose, it should not claim every nearby image as evidence.
That precision matters for trust. A fact returned with an attachment tells the reader that the attachment contributed to its extraction, not merely that it happened to be in the same document. The system also prefers uncertainty to a wrong citation: when the model cannot confidently attribute a fact to an image, it should not attach the nearest one just to fill the field. For charts and tables, the system can create multiple facts from the structured values rather than compressing an entire figure into a caption that silently drops details. This increases memory units, but it preserves more of what a later question might ask.
Attachment provenance has explicit limits. Image-bearing chunks require a model that can read images, and batch retain does not carry attachments. The feature therefore adds evidence quality only when the deployment has the right model and request path. It is not a general guarantee of visual understanding. The useful claim is narrower: when the multimodal path is available, Hindsight can keep a fact's connection to the source attachment that informed it.
Tags provide a different boundary. They do not primarily describe what a fact means; they control which memories can be seen together. A single bank may serve several users, projects, or sessions, so individual memories or documents can carry tags that constrain recall, reflection, and consolidation. A tag such as a project or user scope can prevent a relevant-looking fact from crossing into another context. The same tag can also determine which source memories a higher-level model is allowed to read.
This creates a hidden coupling between retrieval and governance. A tag can make a search narrower, but it can also make a mental model empty if the model requires a tag that its source facts do not carry. Conversely, a broadly shared scope can make a derived observation visible to more callers than intended. Tags are therefore not just filters applied after knowledge has been formed. They influence which facts are allowed to participate in forming that knowledge in the first place.
Bank isolation is the stronger boundary. A bank is intended to keep one user's, agent's, or project's memory separate from another's. The separation means a graph path should not cross banks simply because two entities have the same name. It also means that a useful shared memory design must be intentional: several agents may share one project bank, but unrelated agents should not be combined merely because they use the same tool.
Hindsight adds an opt-in Memory Defense layer for a different class of risk. Before retained content reaches memory units or the stored document body, a bank policy can redact detected secrets and PII or block an item. Partial-block batches can continue with surviving items; if every item is blocked, the retain request returns an error. The control is disabled by default and is not retroactive, so it cannot cleanse memories that were already stored before the policy was enabled. It is a boundary on future writes, not a substitute for access design, source handling, or review of existing data.
The distinction between tags and Memory Defense is important. A tag decides which already-accepted memory may participate in a view or a derived model. Memory Defense decides whether certain incoming content should be stored in its original form at all. Neither is a complete security model. A correctly redacted secret can still be associated with the wrong project if the bank or tag scope is wrong, while a perfectly scoped bank can still retain sensitive content if screening is disabled or a pattern is not covered.
Consider a support system serving several customers from one bank. A customer tag can keep one account's troubleshooting history out of another account's recall and can limit which facts feed a customer-specific model. But if the same support transcript contains an API token, tag scoping does not make the token safe; it only controls who might see it. A redaction policy can remove the recognized token before storage, yet it will not retroactively clean older memories. The two controls protect different transitions in the memory lifecycle and must be evaluated separately.
The connected-memory model is therefore useful because it is selective, not because it connects everything. Entities connect facts about a shared object. Semantic links connect related meaning. Temporal links place events in context. Causal links represent an explanation. Attachments preserve the evidence behind a multimodal fact. Tags and banks decide which of those connections are allowed to participate in a particular view. The next stage is to ask what happens when many connected raw facts begin to point toward the same pattern: how Hindsight can consolidate them into an observation without deleting the evidence that made the pattern visible.