On this page(26)
- What AI Agent Memory Is and Why Agents Need It
- How AI Agent Memory Works: Writing, Storing, and Retrieving Context
- Short-Term vs Long-Term Memory in AI Agents
- The Four Types of Long-Term Agent Memory: Episodic, Semantic, Procedural, and Profile
- Episodic memory: what happened
- Semantic memory: what is true
- Procedural memory: how to do it
- Profile memory: who the user is
- How Agent Memory Differs from RAG and a Larger Context Window
- What Does AI Agent Memory Actually Cost to Run?
- Where Agent Memory Fails: Stale Facts, Memory Poisoning, and Privacy Risks
- Stale facts and weak forgetting
- Memory poisoning
- Path traversal in file-based memory
- Privacy and governance gaps
- Which Agents Need Persistent Memory and Which Do Not
- Tools and Frameworks for Building Agent Memory
- What Changed in Agent Memory This Year
- Frequently Asked Questions About AI Agent Memory
- Do LLMs have memory by themselves?
- Is agent memory the same as fine-tuning?
- Does ChatGPT or Claude remember past conversations?
- Can you delete what an AI agent remembers?
- How is agent memory evaluated?
- Do you need a vector database for agent memory?
- Where to Start with Agent Memory
AI agent memory is the system that saves, organizes and retrieves information across sessions, so a stateless model can act on what it learned earlier. This guide explains how it works, the main memory types, what it costs to run, and where it fails. As of November 2026, Anthropic’s file-based memory tool is generally available, and Claude and ChatGPT both remember past conversations. The evidence on which design works best is still split.
Five figures from 2026 research:
- Forgetting: selective forgetting accuracy fell as low as 7% in multi-hop cases in MemoryAgentBench (ICLR 2026).
- Accuracy: full-context GPT-4o-mini scored 49.8% and verbatim RAG scored 47.2% on LongMemEval-S in the June 2026 MemDelta preprint, a gap that was not statistically significant.
- Cost: a Mem0-based memory system became cheaper than long-context GPT-5-mini after roughly ten turns at a 100,000-token context, according to a March 2026 cost study.
- Security: the MINJA memory injection attack reports a 98.2% injection success rate in its February 2026 version, measured in idealized settings.
- Write quality: weakly written facts failed 24% of the time, against 2% for well-written ones, in the July 2026 Ground Truth First preprint.
What AI Agent Memory Is and Why Agents Need It
AI agent memory is the infrastructure that persists, organizes and selectively retrieves information across interactions, so a stateless language model can act on what it learned earlier. Agents need it because the model itself keeps nothing between calls.
That definition comes from a March 2026 survey of memory for autonomous LLM agents, which argues that memory is what lets a stateless text generator behave as an adaptive agent. The word to notice is selectively. Storing everything is easy. Deciding what to keep, and what to bring back at the right moment, is the hard part.
LLMs are stateless. Each request starts from zero, and the model sees only the text placed in front of it for that call. Whatever it worked out in the last session is gone unless some other system saved it and puts it back.
Anyone who has run a coding agent over several days has seen the symptom. On day two it asks which test framework you use, re-reads files it already summarized, and retries a fix that failed yesterday. The model is as capable as it was on day one. It has no record of day one.
Memory changes three things about how an agent behaves:
- Continuity: it picks up a task where the last session stopped
- Personalization: it keeps a user’s preferences, constraints and history without being told again
- Learning from experience: it reuses what worked and avoids what failed, with no change to model weights
A second 2026 survey, published in ACL Findings, describes the same idea from the model’s side: memory is an external repository that complements a frozen parametric model. The weights hold general knowledge fixed at training time. The repository holds what is specific to this user, this project, this week.

What could a custom AI agent take off your plate?
We build production-grade AI systems that quietly handle the busywork, so your team can focus on the work that actually matters.
I’d treat memory as a system design decision for that reason. It belongs to the application around the model, and upgrading to a newer model won’t add it.
How AI Agent Memory Works: Writing, Storing, and Retrieving Context
AI agent memory works as a loop with three stages: the agent writes information out of an interaction, manages what it has stored, and reads the relevant pieces back into context when a later task needs them. The March 2026 survey of memory for autonomous LLM agents calls this the write-manage-read loop and describes it as tightly coupled to what the agent perceives and does.
- Write: something decides what is worth keeping and records it. In Generative Agents (Park et al., UIST 2023), every experience lands in a natural-language “memory stream”. With Anthropic’s memory tool, as documented in November 2026, Claude itself creates and updates files in a
/memoriesdirectory. - Manage: stored items get consolidated, summarized, corrected or deleted. Generative Agents periodically synthesize higher-level “reflections” from raw memories and store them back, so later retrieval can find a conclusion as well as the events behind it.
- Read: the agent pulls back a small, relevant subset. Generative Agents score each memory on three signals (recency, importance and relevance) and use that score to choose what returns to the prompt.

Where the memories are kept varies. Four substrates cover most systems in practice:
- Plain files: notes the agent reads and edits the way a developer would
- Retrieval stores: records indexed by embedding and fetched by similarity search
- Knowledge graphs: facts linked as entities and relationships, sometimes with the dates they were valid
- Summaries: compressed versions of older history kept in or near the context window
Who controls the loop also differs. MemGPT (2023) treats the context window like RAM in an operating system and lets the LLM decide what to page in and out of external storage, an idea its authors call virtual context management. Anthropic’s memory tool leaves the decisions to the model but runs client-side: the developer’s application executes each file operation and controls where the data lives and how long it stays.
One thing most explanations skip. Retrieval gets the diagrams and the benchmarks, yet the write step sets the ceiling, because a fact that was never recorded, or was recorded vaguely, can’t be found later by any search method.
Short-Term vs Long-Term Memory in AI Agents
Short-term memory in an AI agent is whatever sits inside the context window during the current task, and long-term memory is anything stored outside the window that survives after the session ends. The 2023 CoALA framework, still the standard reference for this vocabulary, calls the first one working memory and sets it against long-term stores.
Working memory is fast and limited. The model sees all of it at once, with no retrieval step, but the limit is more than a token cap. Anthropic’s September 2025 guidance on context engineering says context must be treated as a finite resource with diminishing returns, because recall accuracy falls as the window fills. Anthropic calls the effect context rot.
| Short-term (working) memory | Long-term memory | |
|---|---|---|
| Where it lives | Inside the context window | Outside it: files, retrieval stores, graphs |
| How long it lasts | The current session or task | Across sessions |
| Main constraint | Finite window, and recall falls as it grows | Depends on what was written and what retrieval finds |
| How it’s managed | Compaction and context editing | The write, manage and read cycle |
Agents manage short-term memory by shrinking it. Compaction and context editing both cut what the window carries so a long task can continue, and Anthropic’s memory tool documentation recommends pairing the tool with compaction. One keeps the window usable while the other keeps knowledge safe outside it.
The two layers meet at session start and at each retrieval. Claude Code shows the handoff clearly: as of November 2026, it loads the first 200 lines or 25KB of its MEMORY.md index into every session and reads topic files only on demand. A small fixed slice of long-term memory becomes working memory. The rest waits on disk.
The Four Types of Long-Term Agent Memory: Episodic, Semantic, Procedural, and Profile
The four types of long-term agent memory are episodic (what happened), semantic (what is true), procedural (how to do a task) and profile (who the user is). The first three come from the 2023 CoALA framework. Profile memory is a practitioner’s label for the user-specific layer that consumer assistants made common, and CoALA doesn’t list it as a separate store.
| Type | What it holds | Example |
|---|---|---|
| Episodic | Records of past experiences | Reflexion’s feedback buffer |
| Semantic | Facts about the world and the agent | Zep’s temporal knowledge graph |
| Procedural | Skills and standing instructions | Voyager’s skill library, CLAUDE.md |
| Profile | One user’s preferences and details | ChatGPT’s saved memories |
Episodic memory: what happened
Episodic memory stores specific past events, usually with their outcome attached. Reflexion (2023) is the cleanest example. After a failed attempt, the agent writes verbal feedback on what went wrong into an episodic buffer and reads it before trying again. The next attempt improves with no weight update.
Semantic memory: what is true
Semantic memory holds facts detached from the moment the agent learned them. Facts expire, and that is the hard part. Zep’s January 2025 design handles it with a temporal knowledge graph and a bi-temporal model that tracks when each fact was valid, so an agent can tell a customer’s current address from one that stopped being valid in March.
Procedural memory: how to do it
Procedural memory stores reusable ways of doing a task. Voyager (2023) kept a skill library of verified code, indexed by embedding, and reused those skills in new Minecraft worlds. The everyday version is plainer. As of November 2026, Claude Code’s memory documentation describes CLAUDE.md files: instructions a developer writes once and the agent loads at the start of every session.
Profile memory: who the user is
Profile memory keeps durable details about one person, such as preferences, constraints and recurring context. As of November 2026, OpenAI’s help pages describe ChatGPT’s “saved memories” as explicit entries stored separately from chat history, which is this type in its simplest form. It also carries the most privacy weight, because every entry is about a person.
In practice the types blur. A note saying the integration tests need the local database running first is a fact and an instruction at once. I’d argue the labels are most useful as a reminder that each type needs its own update rule: episodic records get appended, semantic facts get replaced when they change, procedures get verified before reuse, and profile entries stay visible to the person they describe.
How Agent Memory Differs from RAG and a Larger Context Window
Agent memory differs from RAG in who writes the data, and from a larger context window in where knowledge lives and how long it survives. A RAG pipeline reads from a corpus that is maintained outside the agent and stays mostly static between updates. Memory is written and revised by the agent’s own experience. A bigger window only holds more of the current session.
| RAG | Larger context window | Agent memory | |
|---|---|---|---|
| Who writes it | Maintained outside the agent | Nobody, it’s the live prompt | The agent, from its own interactions |
| Changes during use | Rarely | Grows until trimmed | Continuously |
| Survives the session | Yes | No | Yes |
At the mechanism level, RAG and memory overlap. Retrieval-augmented stores are one of the five mechanism families in the March 2026 survey of memory for autonomous LLM agents, so many memory systems are retrieval systems pointed at a store the agent keeps writing to. What separates them is the write path.
A larger window doesn’t give an agent memory, for two reasons. It empties when the session ends. And Anthropic’s September 2025 context engineering guidance reports that recall accuracy falls as context grows, so loading months of history into one prompt makes each fact harder for the model to recall.
On LongMemEval-S, verbatim RAG scored 47.2% and full-context GPT-4o-mini scored 49.8%, a gap that was not statistically significant. (MemDelta, June 2026 preprint)
Does a dedicated memory system beat a long context on accuracy? As of November 2026, the evidence is split. MemDelta changed one component at a time on LongMemEval-S, a version of the benchmark that asks 500 questions over histories of about 115,000 tokens. The ranking between retrieval and full context reversed across models, and agent “self-memory” scored 42%, below plain retrieval. A separate March 2026 study, Beyond the Context Window, found long-context GPT-5-mini had higher factual recall than a Mem0-based memory system on both LongMemEval and LoCoMo.

Read those numbers with care. MemDelta is a single-author preprint that nobody has replicated yet. Vendor-reported gains have the opposite problem: they often bundle changes to the model, the embeddings and the retrieval step, so a headline improvement can’t be credited to the memory design alone. No approach wins everywhere.
What Does AI Agent Memory Actually Cost to Run?
AI agent memory costs extra model calls to write and maintain, and it pays that back through smaller prompts once a session runs long. A March 2026 study comparing a Mem0-based memory system with long-context GPT-5-mini found memory became the cheaper option after roughly ten turns at a 100,000-token context.
The break-even follows from how each approach bills. Full context resends the whole history on every turn, so you pay for the same tokens again and again. A memory system pays once to extract facts, then sends a short retrieved slice. Below about ten turns in that study, resending everything was cheaper.
| Source | Year | Reported figure | Who measured it |
|---|---|---|---|
| Mem0 paper | 2025 | 91% lower p95 latency and over 90% token savings versus full context on LoCoMo | The vendor’s own authors |
| Anthropic context management launch post | 2025 | 84% lower token use in a 100-turn web-search test | Anthropic, internally |
| Sleep-time Compute paper | 2025 | About 5x less test-time compute for the same accuracy on its benchmarks | The paper’s authors |
| Fact-based memory vs long-context study | 2026 | Break-even near ten turns at a 100k-token context | The study’s authors |
The two largest figures are vendor-reported. Mem0’s April 2025 baseline is full context, the most expensive way to run a long conversation, and the paper’s headline says nothing about the gap against plain retrieval. The 84% figure in Anthropic’s September 2025 launch post on context management comes from context editing, which trims the live window instead of storing knowledge outside it.

Three costs rarely make the headline number:
- Extraction calls: Mem0 runs LLM-driven extraction and update phases, and Google’s Memory Bank uses Gemini to extract memories from sessions asynchronously. Every write is inference you pay for.
- Offline compute: sleep-time compute has a model reason over stored context before any query arrives. That lowers test-time compute, but the offline work still runs on your bill, and Letta’s docs label its sleep-time agents experimental.
- Storage upkeep: with a client-side design such as Anthropic’s memory tool, your application holds the files, so file-size caps and expiry rules are yours to build.
Short sessions rarely repay that overhead.
Where Agent Memory Fails: Stale Facts, Memory Poisoning, and Privacy Risks
Agent memory fails in four ways: it keeps facts that have gone stale, it accepts malicious entries, it exposes files outside its own directory, and it holds personal data with weak rules for removal. Persistence is the common cause. One bad write keeps shaping every session that reads it.
Stale facts and weak forgetting
Forgetting is the weakest skill measured so far. MemoryAgentBench (ICLR 2026) tested retrieval, test-time learning, long-range understanding and selective forgetting, and current methods fell short on all four.
Selective forgetting was the weakest of the four abilities, with accuracy as low as 7% in multi-hop cases. (MemoryAgentBench, ICLR 2026)
Time changes the ranking too. Ground Truth First, a July 2026 single-author preprint, found a map-based memory design led at three weeks but recalled only 72% of evicted content at nine weeks, while a provenance-typed graph rose to 90%. Weakly written facts failed 24% of the time, against 2% for well-written ones. Nobody has replicated those results yet. If they hold, a three-week pilot can pick the wrong design.
Memory poisoning
Memory poisoning plants a false record that the agent later retrieves and trusts. MINJA, an injection attack whose latest version is dated February 2026, works through ordinary queries alone. It reports a 98.2% injection success rate and a 76.8% attack success rate.
Those figures come from idealized settings. A January 2026 follow-up on clinical agents working with electronic health records found that pre-existing legitimate memories substantially reduce attack effectiveness. An agent with real history is a harder target. I wouldn’t relax on that basis, since every new deployment starts with an empty store.
Path traversal in file-based memory
File-based memory inherits an old file-system bug. As of November 2026, Anthropic’s memory tool documentation warns about path traversal, where a crafted path such as /memories/../../secrets.env reaches outside the memory directory. The tool runs client-side, so your application executes every file operation and has to validate every path. The same docs flag file-size caps and memory expiration.
Privacy and governance gaps
Rules for removing memories lag behind methods for collecting them. A June 2026 survey of always-on agents, covering 435 works, found the literature focuses far more on accumulating and retrieving state than on governing, recovering or removing it. A survey of long-term memory security, updated on October 30, 2026, recommends provenance, versioning and policy-aware retention as defenses.
Vendors expose some controls as of November 2026:
- Claude: excludes sensitive topics such as health from memory by default
- ChatGPT: offers Temporary Chats that don’t use memory
- Google’s Memory Bank: isolates memories per identity and documents poisoning mitigations
Which Agents Need Persistent Memory and Which Do Not
Agents need persistent memory when the work spans sessions and depends on what happened in earlier ones. Agents that finish in one sitting, with all the context fitting cheaply in the window, usually don’t.
Three kinds of agent clearly benefit:
- Multi-session coding agents: Anthropic’s memory tool documentation describes a coding pattern built on a progress log and a feature checklist, so each new session knows what is done and what comes next.
- Personal assistants: preferences and constraints stated once should hold across conversations without the user repeating them.
- Long-running workflows: a March 2026 cost study found a Mem0-based memory system became cheaper than long-context GPT-5-mini after roughly ten turns at a 100,000-token context.
I’d argue more agents get memory than need it. A single-shot task has nothing to carry forward. A short conversation fits in the window. And when the full history fits cheaply, a memory layer can add cost and risk without adding accuracy: in the June 2026 MemDelta preprint, full-context GPT-4o-mini and verbatim retrieval scored within three points of each other on LongMemEval-S, and an agent managing its own memory scored below both.
Three questions settle most cases:
- Does the task continue after the session ends?
- Does the history outgrow the window, or cost too much to resend on every turn?
- Can you govern what gets stored, including who can read it and how it gets deleted?
A yes to the first two makes the case for memory. The third decides whether you’re ready to run it, and it weighs most in healthcare and financial services, where a stored memory is often a record about a person. Settle all three at the start of any AI agent development project, before anyone picks a store.
Tools and Frameworks for Building Agent Memory
The main tools for building agent memory, as of November 2026, are Anthropic’s memory tool, Claude Code’s CLAUDE.md and auto memory, Google’s Memory Bank, Mem0, Zep and Letta. They differ in approach, in storage model and in who controls the data.
| Tool | Approach | Storage model | Suits |
|---|---|---|---|
| Anthropic memory tool | Claude creates, reads, updates and deletes notes through tool calls | Files in a /memories directory | Teams that must own storage and retention |
| Claude Code memory | Developer-written CLAUDE.md plus auto memory notes Claude writes | A MEMORY.md index and topic files | Coding projects with standing conventions |
| Google Memory Bank | Gemini extracts memories from sessions asynchronously and consolidates contradictions | Memories scoped to an identity, recalled in full or by similarity search | Per-user memory on Google’s agent platform |
| Mem0 | LLM-driven extraction and update phases | Extracted facts, with a graph variant | Long conversations where prompt size drives cost |
| Zep | Builds a graph of facts and tracks when each was valid | Temporal knowledge graph | Facts that change and need a history |
| Letta | Background “sleep-time” agents reason over stored context before queries arrive | Stored context, reworked offline | Contexts that get queried many times |
Data control is the sharpest difference. Anthropic’s memory tool is client-side: Claude requests a file operation, and the developer’s application executes it and decides where the files live and how long they stay. The docs list the tool as generally available with no beta header and as eligible for zero data retention.
Google’s Memory Bank sits at the other end. Extraction and consolidation run as a Gemini-based service, memories are isolated per identity, and Google Cloud’s Memory Bank documentation adds agent-driven CreateMemory writes. For Mem0, Zep and Letta, ask where the store runs and who can read it before you commit.
A few cautions when choosing:
- Maturity: Letta’s docs describe its sleep-time agents as experimental.
- Evidence: the papers behind Mem0 and Zep were written by each vendor’s own authors, so test on your own data.
- Fit: a temporal graph pays off when facts expire, and it is extra machinery when they don’t.
- Volume: handwritten or agent-written files work for a small set of notes, while extraction services exist for histories too large to review by hand.
One advantage of files gets too little credit. When an agent acts on a wrong belief, you can open the file and read exactly what it believes. That makes a bad memory a five-minute fix.
What Changed in Agent Memory This Year
In 2026, agent memory changed on three fronts: consumer chat memory became a default feature with finer user controls, researchers began testing memory systems under controlled conditions, and security and governance work grew into a literature of its own.
Consumer assistants moved toward memory the user can see and edit. Claude’s release notes record three changes between March and August. As of November 2026, OpenAI’s help pages describe a newer “continually updated synthesis” of past chats in ChatGPT, shown with a visible source indicator. Google’s Memory Bank, in public preview since July 8, 2025, documents memory-poisoning mitigations as of November 2026.
| Date | What changed |
|---|---|
| March 2, 2026 | Claude’s chat-history memory reached all users, after a paid-plan launch in September 2025 |
| March 2026 | A cost study compared fact-based memory with long-context models |
| June 2026 | MemDelta tested memory components one at a time, and a survey of always-on agents covered 435 works |
| July 10, 2026 | Claude’s memory moved from a daily summary to individual categorized entries |
| July 2026 | Ground Truth First compared memory designs at three and nine weeks of history |
| August 25, 2026 | Claude added editable Topics and excluded sensitive topics such as health by default |
| October 30, 2026 | A survey of long-term memory security reached its fifth version |
Evaluation got stricter. MemoryAgentBench, published at ICLR 2026, made selective forgetting a tested skill alongside retrieval. MemDelta and Ground Truth First brought controlled comparisons and longer time spans, though both are single-author preprints awaiting replication.
Security research now treats memory as a lifecycle. The October 2026 survey sorts attacks and defenses into six phases: write, store, retrieve, execute, share and propagate, and forget and rollback. It names persistence, statefulness and propagation as the properties that make memory attacks new. Forgetting now has a phase of its own.
Frequently Asked Questions About AI Agent Memory
The most common questions about AI agent memory ask whether models remember on their own, how memory relates to fine-tuning, what consumer assistants keep, and how memory gets deleted, tested and stored.
Do LLMs have memory by themselves?
No. An LLM is stateless: each call sees only the text in its prompt, and nothing carries over once the call ends. The weights hold general knowledge fixed at training time. Anything that looks like remembering a user or a project comes from an application that saved information and placed it back into the prompt.
Is agent memory the same as fine-tuning?
No. Fine-tuning changes a model’s weights, while agent memory keeps information in an external store that can be read, edited or deleted without retraining. Reflexion (2023) showed the difference early: its agent improved between attempts by reading its own stored feedback, with no weight update. The 2023 CoALA framework does count weights as one form of procedural memory, so the two ideas meet at that edge.
Does ChatGPT or Claude remember past conversations?
Yes, both do as of November 2026. Claude’s chat-history memory reached all users on March 2, 2026. OpenAI’s help pages describe two layers in ChatGPT: explicit “saved memories”, and a “reference chat history” feature that draws on past conversations. ChatGPT’s Temporary Chats don’t use memory.
Curious what AI could do for your business?
No jargon and no hard sell. Just a friendly look at where AI fits, and where it doesn't.
Can you delete what an AI agent remembers?
Yes in the major assistants, though deletion can be narrower than it sounds. As of November 2026, OpenAI’s Memory FAQ for ChatGPT says deleting a saved memory doesn’t remove mentions of the same detail in old chats. Claude added editable Topics on August 25, 2026. In a system built on a client-side tool such as Anthropic’s memory tool, your application holds the files, so deletion and expiry work however you implement them.
How is agent memory evaluated?
Agent memory is evaluated with benchmarks that ask questions over long, multi-session histories. LongMemEval (ICLR 2025) tests five abilities: extraction, multi-session reasoning, temporal reasoning, knowledge updates and abstention. LoCoMo (ACL 2024) covers very long conversations, and MemoryAgentBench (ICLR 2026) adds test-time learning and selective forgetting. LongMemEval’s own tests put commercial assistants at 30 to 70% accuracy, but those figures describe 2024-era models, so treat any product ranking built on them as dated.
Do you need a vector database for agent memory?
No. As of November 2026, Anthropic’s memory tool and Claude Code’s auto memory both store plain files, and Google’s Memory Bank can recall by fetching all memories as well as by similarity search. Knowledge graphs and summaries are other options. I’d add similarity search once the store grows too large to load or browse, and start without it.
Where to Start with Agent Memory
Start with AI agent memory by proving the need, building the simplest version you can inspect, and adding infrastructure only when a test says it helps. Four steps, in order:
- Confirm the need: check that the work continues after a session ends and depends on what happened in earlier ones.
- Begin with files: give the agent a small set of notes it can read and edit, and pair them with compaction.
- Set rules early: decide what a well-written entry looks like, when entries expire, and whose memories each agent can read.
- Measure before scaling: run the same tasks with the full history in context, and add a graph or an extraction service only if memory beats that baseline on your own data.
Step four is the easiest to skip. It’s also the cheapest.
AlphaCorp AI can run that baseline test with you and review your memory design as part of an AI integration audit.





