Memory is a design choice.
Not a bigger prompt.

A longer prompt does not make a system remember.

A context window is the material a model receives for one call. Persistent memory is information that the surrounding application chooses to retain, update, and provide again later. A conversation may be visible now and absent from the next request; a stored note may survive across sessions without being relevant to every task.

Memory has a lifecycle: information is selected and indexed, retrieved for a later decision, interpreted in context, updated when it changes, and sometimes removed. Each stage can lose meaning. The LongMemEval paper describes a similar design path through indexing, retrieval, and reading.

Keep the terms separate when designing a system. Tokens and context windows ↗ describe what fits into a model call; context engineering ↗ describes what belongs in that call. Memory is the longer-lived state that may supply part of it.

Facts need time, scope, and a source.

Useful memory is more than a bag of similar paragraphs. It needs enough context to distinguish a durable preference from a temporary request, a direct statement from an inference, one person’s information from another’s, and a current fact from something that has since changed.

Retrieval should be selective. Bringing every stored detail into a new interaction adds noise and can expose information unrelated to the current task. Keep a useful memory small enough to inspect, and retain a path back to its source when the distinction matters.

Remembering the words is not the whole task.

LongMemEval, an ICLR 2025 benchmark paper, evaluates five abilities with 500 questions embedded in long assistant-user histories: information extraction, reasoning across sessions, temporal reasoning, knowledge updates, and abstention. The authors report a 30% accuracy drop on sustained-interaction memory for the systems they tested. That is a finding about those systems and benchmark conditions, not a universal score for every assistant.

These categories reveal different failure modes. A system may find the right sentence but miss that it was superseded; combine facts that belong to different people; fail to connect details from separate sessions; or answer confidently when the needed information was never provided.

Read the LongMemEval paper ↗

Correct recall matters when it guides the next step.

Newer research tests memory in more interactive settings. MemoryAgentBench, introduced in a 2025 arXiv paper and revised in 2026, examines accurate retrieval, test-time learning, long-range understanding, and selective forgetting across incremental multi-turn interactions. Its authors describe these as complementary abilities that existing benchmarks did not cover together.

A 2026 preprint, Mem2ActBench, asks a further question: does an agent apply stored preferences and task state when choosing a tool and filling in its parameters, rather than merely answer a direct recall question? It introduces 400 tool-use tasks for that purpose. Another 2026 preprint, StructMemEval, examines whether systems can organize information as structured records such as ledgers, task lists, and trees. These papers offer different test lenses; none is a universal measure of memory quality.

The practical distinction is simple: test both what the assistant can retrieve and what it does with that information. A remembered detail can improve an action only if it is current, scoped to the right task, and still appropriate to use.

Test corrections, contradictions, absence, and forgetting.

Build a small evaluation set around realistic changes in a conversation:

• Correction: update a preference and check that the old value no longer wins.
• Time: give an instruction with an expiry or date and ask about it later.
• Scope: mention two people or projects with similar details and check that they stay separate.
• Absence: ask about something never provided and see whether the system asks, checks, or abstains.
• Action: use a remembered setting in a tool task and inspect the selected tool and parameters.
• Forgetting: remove or replace a detail, then check whether it returns through another summary or index.

Measure more than exact recall: track outdated answers, incorrect associations, useful evidence retrieved, unnecessary details surfaced, and whether updates carry through later interactions. For an agent, also inspect task success, tool choices, and the review required before a consequential action.

For a broader view of how memory connects to reasoning and agent systems, explore Symploka’s memory and reasoning questions ↗.