a16zInside the Race to Measure Frontier Intelligence
a16z
The Need For Independent Testing 0:00
As soon as a new trillion dollar industry emerges, an independent testing group tends to follow, and that is the role this conversation is about. The clearest example given is Meta's release of Llama 4. On held out private benchmarks the model actually underperformed, yet on the major public benchmarks, where the questions and rubrics are open source, it appeared to show incredible capability. That gap between self reported results and independently measured ones is presented as the core problem driving the need for outside evaluators.
Why Vals Was Founded 1:00
Vals started in early 2024 after its founders, who had backgrounds building benchmarks and evaluations, noticed that public benchmarks were no longer good enough to measure real model progress. Many new capable models were entering the market beyond just OpenAI, and it had become harder than ever to tell what was genuinely new or capable about them. The founders concluded that a third party company was needed solely to build high quality evaluations, so they released their first benchmarks in 2024, a role since echoed by other parts of the industry, including public calls from figures like Demis Hassabis for a broader ecosystem of third party evaluators.
Racing To Evaluate Before Launch 4:00
Before a model's public release, the team works to run tens of billions of tokens of testing without ever becoming the reason a launch gets delayed. Early on this meant the two founders, one named Lynx, pulling all nighters to get results out in time. The company has since built a team and heavy infrastructure, running evaluations in a massively distributed way at the maximum rate limits labs allow, aided by an internal system nicknamed Steve, short for "the economic vals employee," which gradually takes over more of the human labor involved.
Making Fuzzy Human Work Explicit 6:00
Evaluating models is compared to the unresolved problem of evaluating humans, where tools like IQ tests, EQ, or the SAT remain contested and models are notably good at gaming benchmarks. The response is to force fuzzy, informal human judgments into explicit form, such as defining what actually separates an associate from a partner at a law firm, something with no existing formal test. Making these enterprise and real world workflows legible is described as the biggest long term bottleneck, since it determines what signal models can be measured and improved against. A comparison is drawn to the MPAA's shifting, informally agreed line between ratings like R and PG-13, suggesting evaluation norms will likewise stay fuzzy but solidify through shared industry agreement over time. A key early decision was to never sell training data to labs, unlike much of the existing data industry, which has built "gimmick style" benchmarks partly to support data sales, an approach compared to the mixed incentives that produced the Enron auditing scandal.
The Benchmark Catalog And RSI 10:00
Among the most used benchmarks are a finance agent benchmark used by major financial institutions and a coding benchmark called VI codebench, which measures how well models turn a natural language prompt into a full stack web application. A newer, more experimental release is the recursive self-improvement index, built because labs increasingly discuss recursive self-improvement without any shared way to measure it. Since directly having a model train its successor is too costly and slow, the index instead builds proxies covering each stage of building the next model, including pre-training, post-training, and harness level engineering, to see where models show real research capability and where they still struggle.
Retiring Benchmarks As Peaks Are Climbed 12:00
Benchmarks get deliberately deprecated once they become saturated, under an internal motto of "always a higher peak," since the job is to keep constructing new mountains for models to climb, much as labor shifted away from agriculture as economies changed. Benchmarks are also retired because they need to reflect the current state of the world, the same way lawyers must retake the bar or doctors keep certifications current; a benchmark like the legal research benchmark was built to track current case law rather than a fixed, aging snapshot.
Evaluating Agents Over Time 13:30
As evaluation shifts from simple prompt and answer tasks toward agentic work in finance, legal, and coding, infrastructure has had to change to support models running over hours, days, or even weeks, with mechanisms to retry a single failed step rather than redoing an entire task. More complex evaluations now involve fewer total tasks but far richer criteria for judging the output, a shift from something like ImageNet's millions of simple image-to-label mappings toward asking a model to generate 50 full applications and judging them against complex rubrics.
Why Enterprises Face Existential Stakes 16:00
Evaluation is framed as existential not just for labs proving their models improve, but for enterprises deciding how to spend on AI. An anecdote describes a Fortune 10 company giving engineers a roughly $100 a day budget for Cloud Code, later raised to $300, an amount close to an employee's daily salary; because the usage limit resets at 4 p.m., the most productive working hours shifted to 4 to 6 p.m., with a dead period earlier in the day when engineers have run out of budget. This is offered as evidence that intelligence is being misvalued at every layer of the stack, with engineers, companies, and even providers like Anthropic all uncertain about correct usage limits or margins, suggesting token spend may soon rival or exceed salary spend as a company cost.
Val Smith For Enterprise Coding 19:00
Rather than assuming large enterprises can evaluate models internally, since the number of labs, models, and hyperparameter options keeps multiplying, the company built a product called Val Smith. It lets any company use its own GitHub codebase to build an internal coding benchmark, revealing which coding agents will actually perform best and offer the highest return for that specific codebase. Results are often non intuitive: the best OpenAI or best Anthropic model isn't necessarily best for a given repository, and in practice Sonnet can end up costing more than the more capable Opus because Sonnet consumes so many tokens.
Coding As A Template For Other Work 22:00
Coding is treated as a preview of what evaluation will look like across other kinds of knowledge work, since a model capable of strong coding is likely also capable of producing things like PowerPoint slides or Excel-based financial models. The plan for extending evaluation into other industries is to draw on existing repositories of past human work as raw material for building dynamic evaluations, even without having solved the deeper question of what human intelligence itself is.
Vals' Own Token Spending Experiment 22:30
The company ran its own internal experiment, giving its engineering team unlimited access to coding tools for a month to test token maxing. Some engineers spent between one and two billion tokens a day, with one peak day hitting six billion tokens for a single engineer. Doing the math afterward showed the month cost roughly 1.5 million dollars in tokens, about ten times more than that month's employee salaries. This experience led the team to build and use Val Smith internally, uncovering findings such as the coding tool from Cognition, called Devin, being notably token efficient, which shaped their own strategy for controlling token spending going forward.
Token Usage Across Tasks 24:30
You give everyone access to every tool available, but the system autosuggests where to begin a session for any GitHub issue or ticket, which naturally titrates how much compute or token usage gets spent depending on how much intelligence a task actually requires.
Who Should Set AI Policy 25:30
Everyone should be involved to some extent, but policy conversations over the last couple of years have stayed too abstract, with no material grounding for what rules should actually cover. The role of an evaluator, especially early on, is to gather empirical evidence about what models can do and where the risks sit, so that policy discussions have something concrete to build on. Two forces have to be balanced: moving fast enough not to slow down innovation, and making sure the technology serves the broader public interest. The suggested division of labor is that government sets and enforces rules, based on what it fears, such as biohacking or cyber risk, while a competent third party tests whether a model can actually do the harmful thing and whether someone could get it to. Government is well suited to setting rules because it can enforce them, but poorly suited to the ongoing technical work of evaluation.
Briefing Government and Going Global 32:00
Regular briefings are now given to executive and legislative branches on model capabilities and risks, so policymakers can track emerging problems, though decisions about actual policy, such as curtailing release over mental health risks for people under eighteen or biosecurity concerns, are left to them. On the geopolitical side, sovereign AI investment looks inefficient from a birds-eye view, since countries are duplicating data centers and training efforts rather than consolidating them. A comparison is drawn to nuclear arms control, where Reagan's line "trust but verify" applied through flyovers that let one country audit another's stockpile; a shared language of evaluations could serve a similar verification role for AI, especially as concerns shift toward cybersecurity, biosecurity, and eventually recursive self-improvement, where one country or company could pull far ahead in ways others can't observe. Benchmark work is expanding beyond code-level issues like memory leaks toward infrastructure-level risks, such as enterprise cloud or grid systems, to keep pace with where real capability and risk are moving.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
