Why Long-Term Agent Memory Fails at Updates
This series studies long-term memory systems for large-language-model agents: systems that turn an interaction history into state an agent can use later. This opening focuses on the specific problem hidden by the word “memory”: an agent may retrieve a recent fact and still fail because that fact changed, because an earlier version matters, or because the answer depends on several updates at once. The practical question is therefore not only whether an agent can recall, but whether its memory can preserve and revise state well enough to support reliable answers and actions.
Memory is state carried across time
A conventional picture of memory treats information as something that is stored and later looked up. That picture is adequate when a document remains stable and a question asks for one passage. Long-term agents operate under different conditions. A user preference can change. A software interface can be revised. A web task can alter the state of an account. A tool call can produce an observation that changes what the agent should do next. New information may confirm an earlier fact, refine it, contradict it, or add a consequence without making the earlier fact irrelevant.
The memory system must therefore do more than retain content. It must preserve enough identity, timing, and relationship information to answer which state is current, which state existed before, and how one state led to another. That is why the central subject here is not context length by itself. It is the system connecting an evolving interaction stream to persistent state, retrieval, an answering or acting model, and the infrastructure that keeps the state usable.
This distinction matters because a long context is not a complete solution. Passing an entire history to a model becomes expensive and eventually runs into a fixed input budget. Even before a hard limit, reasoning over a long sequence can be uneven: relevant material may be buried among later revisions, similar statements may compete, and the model may not know which version a question refers to. External memory reduces the amount placed in the active prompt, but it introduces a new responsibility: the write path must decide what to preserve and how to represent change before the read path can find it.
That responsibility is easy to underestimate because “update” can describe several different operations. Sometimes a new observation should replace the current value. Sometimes it should be added while the previous value remains important as history. Sometimes it should modify only one field of a larger record, or be linked to the earlier state as evidence for why the change occurred. A system that treats all four cases as append-only storage may preserve words while losing the meaning of the change. A system that overwrites everything may answer the present correctly while making a migration question or an audit of past decisions impossible. The right behavior depends on the identity of the thing being tracked, the time or version attached to the evidence, and the question that will be asked later.
This is also why memory quality cannot be inferred from the final answer alone. A wrong answer may result from an update that was never encoded, a valid record that was summarized without its provenance, a correct record that was not retrieved, or evidence that reached the answering model in a form it could not use. These failures look similar from the outside but require different remedies. The opening model keeps them separate so that later evidence about benchmarks, architectures, and serving systems can be connected to a specific boundary.
The scale of the problem is visible in MINTEVAL, a benchmark built around long, interconnected histories rather than isolated facts. Its four domains include state tracking, multi-turn dialogue, Wikipedia revisions, and GitHub commits. Contexts average about 138,800 tokens and roughly 86 ordered updates, with some reaching 1.8 million tokens. Across the evaluated systems, the reported average accuracy is only about 28 percent, and the best overall result is 33.4 percent. These numbers do not rank every memory system in existence; they show that an evaluation designed around change and interference remains difficult even for systems built specifically to manage memory.
The same fact can have three valid answers
Consider a small revision history. At one point, an article says that a building has eight floors. A later revision changes the value to ten. A question asking how many floors the article states now should return ten. A question asking what the previous revision said should return eight. A third question might ask when the change occurred or how many other facts changed alongside it. The words “how many floors” are not enough to identify the operation. The system must understand the time reference and the relationship among versions.
This is the difference between current-state recall, historical lookback, and aggregation. Current-state recall asks for the latest usable representation. Lookback asks the system to recover an earlier state despite later information that may be more recent, more similar, or more frequently retrieved. Aggregation asks it to find multiple pieces and combine them: order events, count occurrences, compare versions, or perform a chain of reasoning across updates. Each operation tests a different promise about what the memory preserved.
MINTEVAL makes those promises explicit. Its “Simple” questions target the most recent state, while “History” questions target an earlier state after subsequent updates. Its aggregation questions cover ordering, counting, and multihop reasoning. The reported gap is substantial: simple questions average 47.5 percent, history questions 21.0 percent, and multi-target aggregation 26.5 percent. A system that looks competent on the first number can still be unreliable when a user asks what used to be true or what changed across several revisions.
The benchmark’s analysis also separates two failure points that are often blurred together. In a Wiki Revisions analysis, the needed evidence always exists somewhere in the full history, yet only 58.3 percent of cases contained that evidence in the retrieved documents or constructed memories. That corresponds to a 41.7-point loss before the final answering model reasons over the evidence. When the evidence was available, answering errors produced a further average drop of 25.2 points. The exact decomposition belongs to that benchmark and its evidence-checking procedure, but the architectural lesson is broader: a strong answer model cannot recover information that the memory construction or retrieval path discarded or failed to surface.
The same analysis finds that longer lookback distances generally hurt performance. Adding explicit dates or timestamps reduced the measured degradation in the tested full-context and retrieval settings. Temporal markers help because they give similar facts a way to remain distinct. They are not a substitute for a version-aware representation: a timestamp is useful only if it stays attached to the fact and the system can use it when the question asks for a particular state.
Update policy matters as well. In one MINTEVAL ablation, processing a history in larger chunks, and therefore performing fewer memory modifications, generally improved performance for questions requiring historical or aggregated reasoning. Separate analyses of two memory agents found that insertion dominated their operations—87.6 percent for AtomMem and 65.9 percent for Mem-alpha on average. The finding is not a universal instruction to update less often. It points to a sharper issue: when a small revision is expressed implicitly, a coarse system may append another entry instead of modifying the existing state, leaving redundant or conflicting memories for retrieval to sort out.
The memory problem can be pictured as a loop rather than a box. Information enters, gets constructed into persistent state, is retrieved for a query, and supports an answer or action. Maintenance must continue to revise that state. Interference can damage the write path; missing or stale evidence can damage the read path.
A memory system has a write path and a read path
That loop gives this series its operating model. Memory construction decides how raw interaction becomes a record, summary, graph, index, or other representation. Storage determines what kinds of access are possible. Retrieval selects evidence for the current question. Prompt assembly or action connects that evidence to the agent’s next decision. Maintenance handles consolidation, conflict resolution, forgetting, compression, or re-embedding over time. A failure at any one stage can look like “the agent forgot,” even though the underlying causes differ.
The boundaries also determine who effectively owns the state. If the model’s active context is the only store, the model and its context budget own both retention and forgetting. If an index or vector store holds chunks, the construction process owns what becomes searchable and the retrieval policy owns what enters the prompt. If a graph or structured record is maintained, its schema and mutation rules decide whether a relationship survives. In an agentic design, the controller may choose when to write or search, which adds flexibility but also makes behavior variable. None of these choices is automatically better. Each defines what kinds of questions the system can answer, what kinds of changes it can preserve, and which costs are paid on every new interaction or only when someone asks a question.
Maintenance is especially important because a long-lived store does not remain correct simply by growing. New entries can duplicate old ones, conflict resolution can choose the wrong value, and a history that is never pruned can make retrieval noisier and more expensive. Conversely, aggressive compression can remove the provenance needed to distinguish a current fact from a prior one. The system must balance a compact working context against a persistent record rich enough to reconstruct the state that the user actually means. That is an operational version of the title’s claim: the difficult decision is not merely what to remember, but what to change, what to retain, and what to make recoverable.
The next chapters will make the first distinction concrete through revision histories and interference-heavy questions. They will then extend it to agent trajectories, where an action changes an environment and the resulting observation constrains what can happen next. That setting makes causal memory visible: a semantically related fact may still be insufficient if it omits a precondition, an exact identifier, or the state change that made a later action valid. From there, the series will examine how different architectures divide work between writing and reading, and why a fast query can conceal expensive construction or stale updates.
The judgment this subject supports is consequently more demanding than “does the system remember?” A useful memory must preserve the right update, distinguish current from historical evidence, expose the relationships needed for the query, and do so with acceptable freshness and cost. Those are separate properties. Keeping them separate is the first step toward understanding why long-term agent memory fails at updates, not just recall.