20VC with Harry StebbingsThe Future of Inference Market & Model Routing: Commodity or Increasing in Value? | LangChain CEO
20VC with Harry Stebbings
Agent building basics 0:00
LangChain makes tools for building agents, mainly an open source framework that connects a model to tools, data, and computation, plus LangSmith for observability and evals. The hard part is reliability. Models are nondeterministic black boxes, so small changes in input can lead to very different outputs. An agent is also a chain of model calls, not one call, so errors can cascade.
Harnesses and control 3:01
As models improve, parts of the product disappear. Early JSON parsing for tool use gave way to native tool calling, and the team had to keep guessing which hacks were temporary. The durable layer is the harness around the model: context, memory, data, and APIs. You want that layer to stay portable, so you can switch models, avoid lock-in, and control vertical agents where domain experts define verifiers and use cases.
Routing by task context 14:01
Routing is a hot, divisive topic. The split is between gateway routing, which is easy to port across harnesses, and harness routing, which knows more state, prompt cache, and task details. The view here is that vertical agents need harness-aware routing. Open router-style gateways mainly help when you want one API for a chosen model, not deep smart routing.
Evaluation and learning loops 20:00
Routing is not treated as a commodity, because models change too fast and a new model can erase its value. Evals help, but public benchmarks are often maxed out. Internal, domain-specific evals matter more, along with traces. The useful value sits in the context, harness, or model, and continual learning means an outer loop that uses traces to update one of them and then measure the gain.
Reflection remains hard 27:00
Models still struggle with reflection and meta learning. The speaker says it is hard for agents and hard for people too, because we do not fully know what happens inside our own minds. The open problem is not just better models. It is also the shape of the whole system: the inputs, the outputs, the harness, and the workflows.
Memory will become common 29:00
A memory layer for agents is described as badly unsolved but likely to become standard. The speaker says products like Instinct and Muse already show how personalization and follow-up can feel like the system is learning you. Still, memory is early, domain specific, and harder than it looks. The best solutions may need to be vertical, with agents that learn facts within one field and use a harness built for that job.
Agent Spend and Org Design 40:32
A few mistakes have caused huge usage spikes, including a web plugin that kept sending DOM snapshots back on every turn and ran up about $40,000 in a week. That makes spend controls worth having, even with a liberal policy. The harder question is what coding agent spend looks like in two years. The answer depends on org design. Some teams may shrink and use far more agents, but the work still looks human and collaborative, so the spend may simply move to fewer, better-paid people.
Inference and Model Choices 44:00
Agent value already matters most in software engineering, but other enterprise tasks are likely to change too, which is bullish for model providers. Open models are useful, but mostly for routing, guardrails, extraction, and summarization. Enterprises worry about Chinese open models, yet those fears are mostly unfounded if you host them yourself. Inference demand is far bigger than expected, and any solid provider can work. The best bets are inference, infrastructure like Modal or Temporal, and vertical agents with strong teams, speed, and brand.
Execution and investors 55:00
He says being first is not enough. AutoGPT arrived after LangChain’s rise, but speed and execution still mattered, and he has not heard much from it since. He says Jev’s bull case is that it created a new inference and model category and has a team that can execute on that. On investors, he says Benchmark became a third co-founder in practice, while Sequoia helped across recruiting, marketing, launches, and executive hiring.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
