Stanford Online

How AI Agent Memory Works: MemGPT & Memory Types | Stanford CS329Z Lecture 7 with Michael Ryan: summary

YouTube summary12 sectionsWatch on YouTube ↗

This is an AI-generated summary of the YouTube video "How AI Agent Memory Works: MemGPT & Memory Types | Stanford CS329Z Lecture 7 with Michael Ryan" (Stanford Online), made with Samuraize and published by Samuraize. It condenses the YouTube video into 12 titled sections you can read in a couple of minutes, each linking to the moment in the video it covers.

1
Filed under💻 Technology0 comments🍱 Add to trayReport
Study this

How AI Agent Memory Works: MemGPT & Memory Types | Stanford CS329Z Lecture 7 with Michael Ryan

Stanford Online

切

Why memory matters 0:05

Memory keeps an agent from starting over every time you talk to it. Without it, you lose procedure, so the agent forgets how to do repeat tasks. You also lose knowledge, such as a rule to ask before deleting emails or spending money. And you lose progress, so long conversations and long projects drift away from their original goal.

切

Four memory types 7:02

Working memory is the context window, where the agent holds what it is seeing and thinking right now. Episodic memory stores raw traces of past behavior, like tool calls and conversation logs. Semantic memory stores facts about you and the world. Procedural memory stores skills and rules, such as a file that explains how to apply a brand style to slides. These categories are useful, but they are fuzzy and often overlap.

切

Memory can be tailored 14:31

Memory is not fixed to general buckets. You can build domain-specific systems for specific agents and specific facts. The lecture also separates episodic, semantic, and procedural memory from short-term and long-term memory. Working memory is the closer match to short-term memory, while episodic memory is about past events and can sit in either a short or long frame.

切

Where memory lives 17:02

A second split is between parametric and context memory. Parametric memory lives in model weights, through pretraining or fine-tuning. Context memory is added to the prompt window from outside sources. Context can usually override weights, though deeply tuned behavior may resist that. In practice, small fact sets usually stay in context, while procedural skill can be improved by fine-tuning or reinforcement learning.

切

Three storage systems 21:00

The main design questions are what to store, when to trigger storage, and how to store it. The common storage systems here are files, databases, and knowledge graphs. A file system lets the agent manage memories as folders and files. Databases support vector search, SQL, or BM25. Knowledge graphs, like Zep, turn conversations into entities, links, and summaries for retrieval.

切

Graph lookups 29:00

Knowledge graphs differ from databases because they let you traverse links between entities, not just fetch rows or exact text. That makes graph lookup methods, including graph RAG, useful when you want the search step itself to do more work. The speaker treats this as a design choice. In MemGPT, documents are embedded for search, while conversations stay as full text for exact string lookup.

切

Procedural memory 31:00

Voyager stores skills as programs with descriptions, then retrieves them by embedding similarity. When it learns how to mine wood, fight zombies, or craft tools, it saves that code in a skill library and reuses it later for related tasks. The same idea appears in agent workflow memory, where successful web-navigation traces are turned into reusable steps. Skill files make this even simpler: the agent reads a saved skill.md from the file system when it needs it.

切

Tiered memory hierarchy 43:31

MemGPT gives the LLM tools to manage its own memory. The context window holds system instructions and working context, while a first-in, first-out queue holds conversation history. Old messages move out of context into recall storage, where they can be found by exact string lookup, or into archival storage, which works like a vector database for long-term facts and world knowledge. The model can write, replace, and search these stores as needed.

切

Reflection and consolidation 54:31

The paper’s key point is that reflections, not just raw logs, should be stored. In the generative agents example, perceptions go into a memory stream, then retrieval, planning, and reflection shape later actions. A reflection about Isabella planning a Valentine’s party is pulled back as recent, important, and relevant. An ablation shows reflection is one of the main factors behind the best true skill rank.

切

Sleep time compute 58:30

A model can spend idle time reasoning over earlier conversations and facts. In the juggler example, it works through the arithmetic while the user is away, then later answers instantly from what it worked out the night before. The same idea is used to consolidate memory. Chad’s scattered color preferences are rewritten into a single order, with gray first and green last, so later retrieval is easier.

切

Memory evaluation 1:07:00

The lecture then turns to benchmarks for agent memory. Locom is built from simulated long conversations and asks ground-truth questions about them, but it also shows a flaw: even when recall is high, too much stored context can hurt answer quality. PersonaMEim is closer to real use, with changing user tastes over multiple sessions and multiple-choice replies. Memory Arena pushes further by testing agents in live tasks, including group travel planning.

切

Memory evaluation challenges 1:13:00

Benchmarks need realistic, long, hard, and checkable tasks. The speaker notes that memory agents sometimes score worst, though other failures can also matter. The main open problems are deciding what to remember or forget, protecting trust and privacy, and using memory carefully. Poisoned memories and overpersonalized replies can make an agent act in ways you did not want.

AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

Summarize your own YouTube video

Paste a YouTube link, article, PDF, ebook or slide deck and get a summary like this in seconds. Free to try, no sign-up needed.

⚔️ Try the YouTube summarizer

Discussion

Sign in to join the discussion. Sign in

More from the Bento Box

Browse the Bento Box →

With your OK, Microsoft Clarity and Google Analytics set cookies (data goes to the US) so we can see what breaks. Cookie details