All-In PodcastNaveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
All-In Podcast
From Neuroscience To AI Chips 0:00
Naveen Rao describes himself as the opposite of an AI doomer, calling artificial intelligence one of the most transformational technologies humanity has created. He got his first computer in 1978, learned to program as a child, and became an electrical engineer partly because he loved science fiction and wanted to build intelligent machines. He later earned a PhD in neuroscience to study how to make computers intelligent. In 2014 he founded Nirvana Systems, the first AI chip company, at a time when almost nobody talked about AI, and eventually sold it to Intel, where he ran the AI group. After 2020 he built infrastructure for scaling large language models, a business that took off after ChatGPT launched in 2022 and later merged with Databricks, now accounting for a quarter of that company's revenue.
Rethinking The Computer From Scratch 3:00
Rao's new company, Unconventional AI, is rebuilding computing from first principles to make it dramatically more power efficient. The original goal was a thousand-fold gain in power efficiency within five years, but progress has been fast enough that he has revised the timeline to three and a half years. The company is organized top to bottom, starting with theorists holding math and neuroscience backgrounds who develop concepts for moving less information around, then testing those ideas as trained models, and finally building physical circuits, boards, and products.
Why Energy Is Running Out 4:30
Rao points to Google, which processes 3.2 quadrillion tokens per month; even at a conservative 10 joules per token, that adds up to 12 gigawatts, against roughly 40 gigawatts the United States currently devotes to data centers. He estimates the world will run out of available energy for AI within about three years as demand keeps growing faster than energy supply. Data center priorities have shifted over time from floor space to networking equipment to GPUs and now to energy contracts secured first, with infrastructure built around them afterward. Energy already makes up about half the cost of serving a single token on something like ChatGPT.
What Biology Already Solved 7:00
The human brain runs on about 20 watts, a monkey's brain on roughly one watt, similar to a cell phone, and small animals like rats and bats run on milliwatts. A squirrel's brain uses just 8 milliwatts yet executes precise jumps between branches every time, meaning over a hundred squirrel brains could run on the power of one phone. The inefficiency in today's computers comes from moving information around: the human cortex moves about 16 billion bits per second, while a high-end GPU moves nearly 30 trillion bits per second outside the chip alone. Digital computers have worked essentially the same way since the 1940s, built for speed rather than efficiency, and transistor scaling that once drove efficiency gains has largely stopped.
Building A Dynamical Computer 10:30
Rao's approach draws on patterns seen throughout nature, such as flocking birds, ant colonies, and metronomes placed on a shared movable plank that spontaneously synchronize through pure physics. His team applied this idea, called dynamical systems theory, to build an image generation model called UNO, made from simulated oscillators and released open source. They later found that sparsity, removing many connections between elements, made the system more trainable and higher performing rather than less. In January the company began building a physical version of this, taping out a chip design by June, and now has working silicon that generates images using only about 500 nanojoules each, many orders of magnitude more efficient than a GPU. Rao calls this 4D computing, combining three physical dimensions with time, since compute and memory exist in the same element rather than being separated as in traditional von Neumann architecture.
Beating Biology Within A Decade 17:31
Mammalian brains sit within one or two orders of magnitude of the thermodynamic limit for intelligence per watt, while today's computers are about ten billion times away from that limit. Rao expects that within three and a half years his approach can reach the limits of two-dimensional lithography, with the company's broader goal being to beat biology itself and enable computing everywhere, including in robots, within roughly a decade. He anticipates a shift from massive gigawatt data centers toward many small, local, adaptive ones, and invokes Jevons paradox, the idea that cutting costs increases total consumption, to argue that a thousandfold efficiency gain would create the largest market humanity has ever seen. In the discussion afterward, he estimated the technology could reach a full product, a complete data center rack, within about two years, noting that existing AI models would still run on it but would need to be ported at the model level rather than at the lower operations level.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
