Stanford Online

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 6 - AI Agents: summary

YouTube summary9 sectionsWatch on YouTube ↗

This is an AI-generated summary of the YouTube video "Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 6 - AI Agents" (Stanford Online), made with Samuraize and published by Samuraize. It condenses the YouTube video into 9 titled sections you can read in a couple of minutes, each linking to the moment in the video it covers.

1
Filed under💻 Technology0 comments🍱 Add to trayReport
Study this

Stanford CME295 Transformers & LLMs | Autumn 2026 | Lecture 6 - AI Agents

Stanford Online

切

LLM limits 0:06

LLMs know only what was in training. They do not know your private code base unless it appeared before the cutoff date. They also cannot act on their own. They answer prompts, but they do not order a teddy bear or carry out a task. That gap explains why coding has moved from simple prompt-and-reply use toward agents that can take work end to end.

切

Retrieval for context 9:00

A first fix is to add relevant facts to the prompt through retrieval augmented generation, or RAG. You retrieve documents that match the query, augment the input with them, and then generate the answer. But this is not free. Retrieved text counts toward cost, reasoning tokens also count, and the model still has a limited context window and can perform worse with very large inputs.

切

Hybrid retrieval methods 21:30

Semantic similarity alone can miss exact word matches, so keyword search matters too. BM25 compares the exact words in the query and document, which can help when a query uses a term like Cuddly that appears in the text. But keyword search alone also fails when the same idea is phrased differently. The usual answer is a hybrid rank that combines semantic search and BM25, and it often works better than either one alone.

切

Re ranking and metrics 29:02

After retrieval, a smaller set of chunks is re-ranked with a single encoder that sees the query and chunk together. This lets the model use token-level relationships rather than two separate vectors, so it can judge finer details. You may also add context to chunks so they stand on their own, and then evaluate the final ranking with metrics like NDCG, reciprocal rank, precision, and recall at K. For code, exact matches are often better, so grep is commonly used.

切

Tool calls in practice 44:30

Tools let the model act, not just answer. For finding a teddy bear near you, the model first predicts a GPS lookup, then a backend search around those coordinates, and only then turns the result into a reply. The same pattern fits web search, weather, stocks, code bases, calculator work, plot drawing, and sending email. The action can be quick, but the model must wait for the backend to finish.

切

How tools are taught 57:30

Tool use used to be trained with examples of the call and the final wording, but that is less common now. Better tool design matters more: clear names, documented inputs, and structured outputs like enums or classes. Prompting still helps, and a separate model can refine tool descriptions by comparing generated answers with the target answers and iterating until they match more closely.

切

Tool conflicts 1:03:31

More tools can create conflict. A tool may match the user’s query and still not be the right one. If you ask for a teddy bear’s morning routine, one tool may retrieve a document and another may also fit. That overlap is harmless when both answers mean the same thing. It becomes a problem when the same words can point to different intents, such as finding a file named teddy bear or asking where the bear physically is.

切

Selection and caching 1:08:00

Tool selection narrows the set before the model sees it. The choice can use conversation history, past user style, user profile, and whether the user has enabled a tool. A small routing model is enough for this. Too many tools also hurt performance by swelling the context. MCP standardizes tools, prompts, and resources across LLM providers. Prompt caching then keeps fixed instructions at the top so repeated tool descriptions do not keep costing full price.

切

Agents and harnesses 1:23:31

The harness around a model includes tools, action order, compaction, memory, and prompting. The model and harness work together, like an engine and the rest of a car; a weak model cannot be rescued by a strong harness, and the reverse is also true. The React pattern, short for reason and act, runs as observe, plan, act, and observe again until the system reaches a useful state. That loop is learned in training, so it becomes a native skill. Agents can also speak through shared protocols such as A2A, which defines skills and functions other agents can call.

AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

Summarize your own YouTube video

Paste a YouTube link, article, PDF, ebook or slide deck and get a summary like this in seconds. Free to try, no sign-up needed.

⚔️ Try the YouTube summarizer

Discussion

Sign in to join the discussion. Sign in

More from the Bento Box

Browse the Bento Box →

With your OK, Microsoft Clarity and Google Analytics set cookies (data goes to the US) so we can see what breaks. Cookie details