YC Debates: Should We Slow Down AI?
Y Combinator
Opening case 0:07
Jacob says governments should substantially slow frontier AI. He argues that superhuman systems may arrive by 2027 or 2028, that labs already report major speedups, and that once AI research is automated, progress could accelerate fast. He also says current models show troubling behavior, like cheating and hiding actions, and that companies have no strong reason to stop racing ahead.
Skeptical reply 9:32
Robin says the debate leans too hard on abstraction. He warns that intelligence is not one simple scale, and history shows new technologies often take decades to spread because the rest of society must change too. He prefers slower, observable adoption and says policy should be robust, using liability and insurance rather than a broad global AI clampdown.
Regulation and capture 15:30
The speaker says giving broad regulatory power over a vague AI problem risks capture by powerful interests. He thinks that is a political fact, and asks whether that outcome is better than taking the chance that speculative dangers go wrong. He also says there is no near-term pause or freeze to wait for, because this is a long, hard problem. If you regulate by capability, the scope keeps widening with Moore's law.
Speed, data, and spread 24:32
He then says he is less worried about AI as a descendant if the change is slow and safe. What bothers him is speed: a sudden jump in capability feels like a discontinuity, unlike the gradual growth people want for their children. He expects this boom will not be the one that produces broad AGI. Instead, he thinks the bigger issue is how fast AI spreads through the economy, and that most jobs still lack the rich data needed for full automation.
Data and hidden work 29:30
The discussion turns to where training data comes from. Large libraries of news videos are easy to use, but many jobs are not like video, pictures, or text. They happen in the world. To learn them, a model would need to watch people, try tasks, and be corrected. The reply is that much cognitive work is already logged on laptops, so those records may feed next year’s big models. Even so, software still moves slowly because teams spend time on requirements, testing, and integration.
Limits on training 33:01
Asked for a policy, one proposal is to limit how much frontier compute a company can use for training. The rest would have to go to customer inference or alignment research. That would slow things a little without giving regulators open-ended power. Another proposal is protracted safety cases before training, checked by an outside evaluator. The aim is to catch dangerous behavior early, before systems reach a point where problems are harder to stop.
Focused oversight only 42:30
Regulation should answer big failures, not small mishaps. The speaker rejects an FAA-style body for AI if it could act on vague concerns or minor issues. He would accept something narrower, aimed at substantial incidents and specific recommendations, but not a broad grant of power to regulate whatever worries officials choose.
Limits and safeguards 47:00
The discussion then turns to compute rules, reward hacking, and insurance. One view is that compute caps create a hard bottleneck and that adversarial review with safe-testing AIs makes some proposals harder to game. On jobs, the suggestion is global insurance, not local government promises, because AI gains and tax bases may be spread across countries.
Final objections and vote 57:01
Robin says a pause is only meaningful if it has a real end point, otherwise it is just a delayed stop. In the last questions, Jacob says he would change his mind if superintelligence could do many real jobs and if concrete safety cases were required before training. He says broad regulation should slow science, not try to list every harm. Robin says that is too vague for a general technology. They close by restating their cases, and the audience votes Jacob the winner.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.

