Agent Memory

Models have no memory between calls. Everything that feels like memory is retrieval, and designing what to store is the actual problem.

A model has no memory between calls. None. Every request arrives fresh.

So every memory feature you have used is retrieval: something was stored outside the model and put back into the context window on a later request. The design question is not how does the model remember but what gets stored, and what gets retrieved when.

Four kinds

They serve different purposes and have different lifetimes.

Working memory — the current transcript. The agent loop’s accumulating message list. Lasts one task, bounded by the context window.

Episodic memory — what happened in past sessions. “Last week you helped me debug the auth service.” Retrieved by relevance to the current situation.

Semantic memory — durable facts. “The user prefers TypeScript.” “Production runs in eu-west-1.” Small, stable, often loaded on every request.

Procedural memory — learned approaches. “Checking the migration log first usually finds this class of bug.” The hardest to do well and the most valuable when it works.

Most systems need the first three. The fourth is largely still research.

Working memory is the immediate problem

Long agent runs exhaust the context window, and the whole transcript is resent every iteration — so cost grows quadratically. Four approaches, in increasing order of effort:

Truncate. Keep recent messages, drop old ones. Trivial, and silently loses the goal statement if you are not careful. Always preserve the task description explicitly.

Summarize. Periodically compress old history into a summary. Extends the horizon, loses specifics, and the compression is lossy in ways you cannot predict.

Compact tool results. The highest-value fix by a wide margin. A tool returning 10,000 tokens where 200 would do poisons every subsequent iteration. Return identifiers and let the agent fetch details on demand.

Externalize. Have the agent write findings to a file or scratchpad it can re-read. The transcript stays short; state lives outside it. More engineering, and the only approach that genuinely scales to long tasks.

Cross-session memory

The pattern that works is unglamorous: store memories as text with embeddings, retrieve the relevant few per request, inject them into context.

It is RAG over the agent’s own history, and every retrieval concern applies unchanged — chunking, hybrid search, reranking.

Three decisions determine whether it is useful.

What to write. Writing everything makes retrieval noisy. Two options: have the model decide what is worth remembering, which is flexible and inconsistent; or write on defined triggers — stated preferences, corrections, resolved problems — which is predictable and misses things. Trigger-based is the better default.

When to retrieve. Semantic facts are small enough to load every time. Episodic memories should be retrieved by relevance, and retrieving too many is worse than retrieving none, because irrelevant memories dilute attention and occasionally get used.

When to forget. The step everyone skips. Memories go stale, contradict each other, and accumulate. Without expiry or conflict resolution, an agent’s memory becomes actively misleading — it will confidently apply a preference the user changed months ago. Timestamp everything, prefer recent on conflict, and expire what has not been useful.

Where it goes wrong

Stale memories applied confidently. The most damaging failure. A model given a memory in context treats it as current fact.

Contradiction. Two memories disagree and nothing resolves them. Store timestamps and prefer recency.

Retrieval noise. Injecting ten marginally relevant memories degrades output. Fewer and better.

Privacy accumulation. An agent that remembers everything builds a durable profile of the user. That has real data handling consequences: users should be able to see what is stored and delete it, and memory content should be treated as personal data.

Memory as untrusted input. A memory containing text that resembles instructions is a prompt injection vector — particularly if memories can be written from tool results or user content.

Practical shape

For most applications: a short list of explicit semantic facts loaded every request, plus retrieval over episodic memories when relevant, plus aggressive compaction of working memory.

Start with the simplest version — a stored list of user preferences — and add retrieval when you can point to a specific failure it fixes. Elaborate memory architectures are usually solving a problem the application does not have.

What to remember

• Models have no memory; everything is retrieval into the context window.
• Four kinds: working (current transcript), episodic (past sessions), semantic (durable facts), procedural (learned approaches).
• Working memory: compact tool results first, then externalize state to files.
• Cross-session memory is RAG over the agent’s history — decide what to write, when to retrieve, and when to forget.
• Stale memories are the worst failure mode; timestamp everything and expire aggressively.

Next: Context Engineering