AI@GSB Perspectives: Infrastructure with Kaushik Shirhatti, VP of AI Factory NVIDIA
Stanford Graduate School of Business
AI factories defined 0:05
NVIDIA is framed as serving AI by making hard systems feel easy, seamless, and effortless. The speaker says the real customers are not the named companies, but the users, machines, and agents that rely on AI to work faster and better. Those users include a clinician using an AI scribe, a researcher, a teacher, students, and even a distracted parent using agents to arrange a birthday party.
The three hard problems 5:02
An AI factory takes data and energy as inputs and turns them into tokens, intelligence, and business outcomes. Building it means dealing with three main problems: deciding which projects matter most, finding and cleaning the data, and choosing the right infrastructure, security, and cost plan. The point is not just design and deployment, but time to first token, time to first train, time to first inference, and time to wow.
Tokens and business value 18:01
Tokens are treated as the bridge between AI output and return on investment. The speaker breaks token economics into four parts: utility, demand, supply, and monetization. A million tokens can mean very different things depending on the work, whether it is research, productivity, or something less useful. Enterprises must map use cases, forecast growth, decide whether to rent or build, and think carefully about governance, security, and model routing.
How AI factories work 23:01
AI factories come in two broad forms. Giga-scale builds serve outside customers, while enterprise setups may start with 16 or 32 GPUs and grow. Most companies will use a hybrid pattern: some cloud and some on-prem, plus both foundation models and open-weight models. The factory rests on compute, networking, software, and an ecosystem of partners. Success depends on the handoff across all four, not on any one piece alone.
Agent workflow basics 35:00
Agentic AI is laid out as a simple loop. You start with a prompt, then use context, memory, observation, reasoning, and action. The point is made through a grocery run for Takis at Safeway, where the wrong choice is corrected after feedback. A useful agent also needs tools and skills. It must connect to email, Slack, documents, repositories, and CUDA libraries, or it cannot do much.
Security and AI factories 38:02
Memory and compute are spread across CPU, DPU, network, and other systems, so security has to come first. The agent should be governed in a sandbox so you know what it can see and do. That same framing leads to the idea of an AI factory. At Nvidia, adoption began with shadow AI, then moved to governance, strategy, and heavy token use across the company.
Enterprise advantage 52:30
AI natives move fast because they start with no baggage. Legacy companies can still win because they own the data. The hard part is not interest in AI but the silos, compliance rules, and data sovereignty issues that keep data apart. If you help solve those problems, the enterprise can do more than a startup can do.
Tokens and daily use 55:00
He says companies should begin with tokens as the basic unit, then move up to task cost, project cost, and ROI once they are more mature. Token per watt is useful in a factory setting. He also says most waste comes from a few big engineering projects, not casual employee use, so people should have access to tools and experiments while cost is managed through the right strategy.
Try new model choices 1:09:30
He urges you to step back and ask what your team should stop doing itself and what AI should do instead. That means mapping the workflow on a whiteboard and changing it with care. He also tells you to use Stanford’s access to different models, including foundation models and open models, and stay curious.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

