a16zWhy AI Can Write Code But Can’t Keep It Running (Yet)
a16z
Background And Early Work 0:00
Jad describes moving from Stanford networking work into security and systems work. He built hardware versions of SHA-3 implementations, then grew tired of networking as a field that only moves bits around. A postdoc at MIT led to application security research and a startup idea about rolling back exploited systems to a clean state, but the work proved too early for the real world.
Greer And AI 5:31
He then joined AppDynamics, moved with its growth from 200 to 2,000 people, and later came to Greer after a short stop at Cisco. At Greer, the team first focused on cutting telemetry and observability costs, because companies were really spending that money to buy reliability. As systems shifted toward microservices, observability data ballooned and the extra reliability from each new dollar flattened out. Greer now strips out signal upstream, cuts cost sharply, and feeds those signals into AI systems that help troubleshoot and fix problems.
How AI Changes Engineering 11:00
On engineering teams, AI started as autocomplete, then grew into something that good engineers learned to use well. He says the best engineers adapt their workflow, learn the harnesses and models, and train the AI as much as it trains them. In his view, the gap between average and strong engineers has widened, because the work still depends on a person who can hold the system’s complexity in mind and break it into pieces the model can use. For their own distributed systems work, the models still need a human to explain the query semantics before they can build the translation layer.
Syntax and semantics 15:03
Semantics are the meaning or behavior of a thing. Syntax is the code you type, the shape of the words, or the characters on the page. A dog and a puppy may point to the same thing, but they still have different syntax. In computer terms, syntax is the code itself, while semantics is what the machine actually does when it runs it. Some behavior cannot be known from syntax alone, so you have to test it.
Daily coding workflow 18:01
The speaker says they use Cloud Code, Windows with WSL, and Opus 5.5. WSL lets them run Linux tools inside Windows, and they work mostly from the command prompt. Their recent change is to have Claude spin up command-line subagents that keep a persistent session. They usually run three or four session windows at once, because more than that makes it hard to keep track of the work.
Faster production delivery 22:01
Before starting implementation, the speaker has the agent write a PR doc and design doc, then reviews the plan with tech leads. They check the algorithms, the split of work, and how the pieces fit together, then let the agents parallelize the tasks. One recent data lake feature, called auto-promotion or shredding, took about 10 days to build. The same work would have taken about six months before, which they describe as a tenfold to twentyfold gain.
Operational limits 25:01
The hard part is not just writing code for a distributed system. It is knowing what happens at runtime, such as availability, correctness, traffic, and memory use. The speaker says models can improve query code when they are grounded in benchmarks and real query data, but they still do not know how much memory to give a system. For that, you still need experiments in production, because real systems do not behave exactly as expected and incidents still happen.
Change Detection First 30:30
The system does not start from a late alert and work backward. It starts by looking for what has changed in the environment, using large-scale clustering and log scanning to find something new or out of pattern. When it sees messages like exhausted connections, memory limits, or empty buffers, it treats them as early warnings and checks whether they point to a real issue.
AI on Operational Clues 35:32
The predictive part is separate from the AI. A model scans huge volumes of logs, finds a new pattern, and lifts it up to the agent, which then reasons through hypotheses, checks code, and asks whether the signal makes sense. That approach has already caught cases in customer systems that were later ignored and turned into production incidents.
Limits And Hope 36:01
Query optimization feels close to being handled by AI, but running a full distributed cluster does not yet. The case for progress is that operations can be reduced to visible signals, changes, and a workflow for debugging, even if the space of failures is large. Still, the speaker stresses that operations and security are harder than code because they are full of rare states and, in security, an active adversary.
Operations Are Harder 45:01
Operations is treated as a much tougher problem than code. Systems have more possible states, random outside load, silent production changes, and deadlocks that come from how pieces interact. It also involves more people, since knowledge is spread across teams and the work is about connecting those parts.
Digital And Real Worlds 53:34
AI is said to work best where the solution space matches the training set, like code or images. The natural world is different. Self-driving cars are given as a case where it took 20 years and $170 billion, and still ended with worse unit economics than Uber. The point is that the real world is harder, slower, and more tied to cost.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
