Stanford Online

Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey: summary

YouTube summary7 sectionsWatch on YouTube ↗

This is an AI-generated summary of the YouTube video "Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey" (Stanford Online), made with Samuraize and published by Samuraize. It condenses the YouTube video into 7 titled sections you can read in a couple of minutes, each linking to the moment in the video it covers.

1
Filed under💻 Technology0 comments🍱 Add to trayReport
Study this

Vibe Coding, AI Risk & AI Evaluation | Stanford CS193T Lecture 6 with Cynthia Bailey

Stanford Online

切

Vibe coding basics 0:05

Vibe coding now means using English as a programming language, from playful experiments to more serious agentic engineering. A useful model is to use AI once to build a tool, then use that tool instead of a full LLM each time. That can save compute, money, and effort when you need the same check or update again and again.

切

Three-step workflow 9:00

The process has three steps: prepare a detailed scope, give it to a fresh coding agent, then review the result with another fresh agent. The scope should include the task, the testing plan, and any constraints from the larger code base. You stay in the loop by asking follow-up questions and by using tests and skeptical review.

切

Model Risk and Bias 21:31

Past grading data can look solid and still fail when students start writing comments meant to fool the system. A tool can pass thousands of historical cases and still miss new behavior after deployment. The same concern applies when a model can keep learning in the field, because its weights may change and its behavior can drift into harm. Some experts therefore argue that AI weapon systems should not be allowed to keep learning after release.

切

Testing Decisions Carefully 28:00

A company that wants to fire people after 90 days may test an AI on old records and see perfect results, but the test can hide survivorship bias. If you only observe employees who were kept, you lose the counterfactual for the ones fired. One fix is to randomly spare some recommended firings so you can compare outcomes. That kind of problem is why NIST’s AI risk framework stresses mapping, measuring, managing, and changing organizational culture.

切

Data handling and Stanford rules 42:02

The talk opens with decommissioning data responsibly. A startup story shows what happens when a business leaves sensitive files behind, including mortgage records with customer financial details. Stanford’s AI policies then come into view. If you handle campus data as a TA, in a lab, or in a club role, you need to know the data classes and which AI tools are approved for each one.

切

Evaluation and ground truth 46:30

The policy’s plainest rule is simple: do not put into generative AI data about others that you would not want used about you. That same check is extended to cameras, recommendation letters, training data, pricing, pay, and edited photos. The lecture then turns to evaluation. Model evals ask whether the AI hallucinates, shows bias, or uses tools well, and often use another LLM as judge.

切

Testing AI outputs 1:02:00

Ruha Benjamin’s point is that crime maps often miss what police never record, and that app makers sometimes rely on convenient data instead of real ground truth. The talk then turns to Twitter’s 2020 smart cropper, which kept choosing Mitch McConnell over Barack Obama, prompting public tests and a call for you to evaluate AI yourself.

AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

Summarize your own YouTube video

Paste a YouTube link, article, PDF, ebook or slide deck and get a summary like this in seconds. Free to try, no sign-up needed.

⚔️ Try the YouTube summarizer

Discussion

Sign in to join the discussion. Sign in

More from the Bento Box

Browse the Bento Box →

With your OK, Microsoft Clarity and Google Analytics set cookies (data goes to the US) so we can see what breaks. Cookie details