Open Models Are Collapsing The Cost Of AI
Y Combinator
Enterprises shifting to open models 0:00
Jeffrey Morgan, co-founder and CEO of Ollama, explains that the biggest trend right now is enterprises moving toward open models, meaning AI systems whose underlying code and weights are publicly available rather than locked inside a single company's product. This shift spans both US and Chinese-origin models and is driven mainly by coding agents, which are AI tools that can write and execute code on their own, and by AI assistants used for everyday work tasks like OpenClaw and Hermes.
Cost is the entry point, not the goal 1:30
Ollama, which Morgan describes as the easiest way to run open source AI models locally and in the cloud, is used by 9 million developers and 85 percent of the Fortune 500. Morgan says cost is the biggest pain point that open models solve for businesses, but the deeper motivation is control: companies want to customize AI for their own needs, and lowering cost is what lets them get there. He points to a case from AT&T, reported in The Information, where the company has already shifted 40 percent of its token consumption to open models, mostly US and European ones so far, while also evaluating Chinese models.
The OpenClaw token explosion 3:30
Morgan shares a graph tracking token usage per developer on Ollama's cloud, which shows two sharp jumps this year. The first came early in the year when open models like Kimi, GLM, and MiniMax became capable enough to power coding agents. The second, much larger jump came in April with the rise of OpenClaw, which let non-developers hand off entire hard problems to an open model and let it work through them, consuming far more tokens as it decided what tools to use and what data to fetch. Context windows, the amount of text a model can consider at once, grew from 128,000 tokens to over a million during this period. Per-developer usage rose roughly fivefold from that single jump, while overall cloud usage grew 150 times since the start of the year.
From fine-tuning to off-the-shelf models 5:02
Morgan notes that in 2024 and 2025, most large open models used at scale were custom fine-tuned versions, meaning companies took a base model like DeepSeek or Kimi and retrained it for their specific use case, the way Cursor famously did. Serving open models straight out of the box, without that extra tuning step, only became common at the start of this year.
Faster release cycles change the calculus 5:30
Morgan observes that open model release cycles are accelerating, with three iterations of the DeepSeek Flash model appearing in a single summer where updates used to come roughly every six months. This faster pace makes custom training harder to justify since a new model can quickly outdate the old one, though better tooling is helping teams that still want to fine-tune keep up.
Safety concerns and security use cases
As AI safety debates slow progress at frontier labs, open-weight models keep advancing, illustrated by the GLM53 model's impressive cybersecurity capabilities. Morgan says the main obstacle to enterprise adoption of Chinese-origin models is safety and security rather than capability, and if that's solved, businesses in the US and Europe are largely open to using them. He also points out that some open models, unlike closed ones such as Claude, will actually perform penetration testing and security research tasks that closed models refuse to do, and that Hugging Face itself had to use open-weight models to detect a hack.
What a day-zero model launch requires 8:31
Because Ollama is a major distribution channel, model developers often coordinate launches with them in advance. Morgan describes the playbook for a successful launch: making sure the model works in the right inference engine, which can take multiple weeks to get fast and accurate; finding the right harnesses, meaning the software frameworks or SDKs developers use to interact with the model; and running benchmarks against the final version to confirm it matches what the lab's research team specified. He gives the example of DeepSeek's first multimodal model, which required updating the DeepSeek harness itself to support new capabilities. Much of this work comes together only in the final 24 hours before release, making it, in his words, generally a fire drill.
Ollama as an operating system for models 11:30
Morgan compares Ollama's role to an operating system, gluing together three pieces: the harness or SDK used to interact with a model, the packaged and reliable model itself with enough cloud capacity, and the underlying hardware and providers, including partners like Nvidia and Apple's silicon stack. He notes his team's background is mostly in classic infrastructure companies like VMware and Docker rather than AI research, and that the same old compute problems are reappearing in the world of AI inference.
The stack has more than five layers 13:01
Referencing Nvidia's Jensen Huang's idea of a five-layer stack, apps, models, infrastructure and inference, chips, and energy, Morgan argues open models actually involve more hidden layers, including model orchestration, developer APIs, and the space between models and applications. He agrees these hidden layers could become their own standalone companies, citing an Anthropic platform team talk that identified three big problems: knowledge, meaning connecting a company's data to the model, coordination among sub-agents running locally and in the cloud, and execution, particularly the compute demands of running agents in sandboxes.
Open models will dominate token volume 18:02
Morgan predicts that within businesses, 80 to 90 percent of tokens processed will eventually come from open models, even though that might only represent 10 to 20 percent of total AI spending, since open models are becoming cheaper to run. Frontier closed models will still handle the hardest tasks where top researchers focus their work, while open and closed models increasingly collaborate, similar to how cloud computing ended up blending proprietary services like AWS's DynamoDB with open alternatives like Postgres.
Local models are catching up fast 21:01
Morgan highlights that current consumer hardware runs 20 to 40 billion parameter models well, and sometimes up to 128 billion parameters, citing that Qwen 3.8's 38 billion parameter model reportedly performs as well as Opus 4.6 for coding on a mid-range MacBook. Local usage shows a close split between US and Chinese models, including Llama and Google DeepMind's Gemma models, while cloud-hosted coding agent usage is dominated almost entirely by Chinese models. Local models work well for simpler tasks like document processing, while harder problems like writing code tests still favor large cloud models, leading many businesses toward a hybrid setup that routes easy tasks locally and hard ones to the cloud.
Desktop Hardware For Running Models 24:31
You can now run more than a frontier-level model at high speed on hardware that costs not far off from a classic workstation computer, something in the range of two to three hundred thousand dollars for serious setups, though even smaller options exist. Many customers, including banks and industrial businesses, already have Nvidia workstation GPUs like the RTX A6000 on every engineer's desk, and the next generation of that hardware, shown at GTC, includes the DGX Spark, which sits on your desk with 128 GB of unified memory and can run models in the 20 to 120 billion parameter range. You can chain several of these together over a fast network link, almost like a miniature data center rack on your desk, similar to how people already stack Mac minis, and this points to a renaissance for personal desktop computing, with Apple's MLX project and Nvidia's DGX Spark both offering mature ways to run large language models locally.
Local Compute May Return 26:30
Ollama's own journey started local, moved to the cloud where coding agent demand currently lives, and is expected to swing back to local devices once hardware catches up. The vision is that when a GB300-class chip sits on your desk, the fastest coding loop becomes as quick as running your tests or making a code edit, recapturing the near-instant feel of the original GitHub Copilot autocomplete. The expectation is not a full return to local-only use but a combination where local and cloud compute are used together.
GPU Supply And Pricing Volatility 27:00
Getting access to the latest B200 and B300 GPUs needed to run frontier models is very hard for startups, and prices are moving quickly amid high supply and demand volatility. Ollama has partnered with several providers to pool GPU access so it can keep up with demand, which involves constant decisions about which model runs where, how fast it needs to be, which region it sits in, and what latency customers will experience. Products like OpenRouter, Ollama, and the Open Code project let an end-user developer get access to this compute without having to negotiate GPU prices or plan a 24-month hardware forecast themselves.
Cheap Flash Models As Workhorses 29:00
For a startup founder trying to stretch a limited budget, the advice centers on a new class of ultra-low-cost models like DeepSeek Flash, which are cheap both per token and per task. The goal is returning to something like the early ChatGPT experience, where nobody worried about counting tokens because usage felt unlimited, though reaching that again for open models requires architecture built specifically for high-volume use. Open models have closed the intelligence gap with closed frontier models to within roughly three months, and the next challenge is extreme efficiency; DeepSeek Flash is already the fastest-growing model on Ollama's cloud, good enough for around 80 percent of tasks while being fast and cheap, making it a workhorse model that can also be chained together with others to solve harder problems through orchestration.
Orchestration Over One Giant Model 31:31
Contrary to earlier predictions of a single giant "god model" handling everything, the more common pattern turns out to be composing smaller, cheaper, task-specific models together, which is more repeatable, more trustworthy, and affordable. Most customer use cases reach a level of intelligence that is simply good enough, after which they don't need to reach for the most powerful model, though some use cases genuinely require frontier-level power. Open models like those from Zhipu AI's GLM and the Kimi model have already hit frontier performance in specific areas, such as web development, turning the competition from a simple gap into a real head-to-head contest.
Model Origin And Security Concerns 33:31
Customers increasingly care not just about where a model was trained but where and how it is run, whether in a secure environment, and how it communicates, since models go through phases of sounding more robotic or more friendly. Some customers care only about where a model runs, while others care specifically about where it was trained, especially for mission-critical uses, such as a real case where Llama powers analytics at a power plant in Finland to detect surges and keep the lights on. On the risk of a "Manchurian candidate" scenario, where a foreign-trained model might be secretly compromised even if hosted domestically, no known cases were mentioned, and it was compared to long-standing supply chain risks in open source software, where thousands of dependencies already create potential points of failure that IT and security teams at large enterprises are well practiced at screening for.
From Docker Roots To Ollama 36:30
Ollama's founders previously built Docker Desktop, which gave them a strong sense of developer experience, but the company spent its first two years searching for the right problem to solve, initially pitching Kubernetes-focused ideas to YC in Winter 2021 under a concept close to "Docker Desktop for Kubernetes." They pivoted multiple times, including an unspoken pivot toward desktop security for developers, before Llama's release in 2023 finally clicked when they tried running the model themselves and found it both hard and impressive to get working. The name Ollama does not come from an existing model but from combining "LLM" with an approachable animal mascot concept, similar to how Docker had its whale.
Raising From Benchmark Pre-ChatGPT 39:01
The company partnered with Benchmark in 2022, before ChatGPT existed, with a pitch centered less on a specific product and more on the team and their history, since the investor Peter had also backed Docker's Series A. The two years of searching for the right problem were genuinely frightening, especially with a team of more than ten people staying loyal through several pivots, and clarity only came when the team gathered in Toronto, decided to start from scratch, and identified two real problems worth solving: building a seamless gateway to access any model, and using their systems expertise to make open models easy to run.
Two Week Sprint To Launch 43:00
The team gave themselves two weeks to launch the first version of Ollama, and the release lined up with the arrival of Llama 2, prompting an immediate decision to ship. That two-week burst of action produced more traction than two years of overthinking had, and Ollama's GitHub project reached 100,000 stars faster than Docker or Kubernetes had. Adoption spread quickly from hobbyists tinkering with local models to roughly 85 percent of the Fortune 500 within about two years, a pace compared to a homebrew-computer-club-to-mainstream leap that took the PC around a decade but happened here in something like 12 to 18 months, largely because open models were free and could be run anywhere without needing permission, making the jump from hobbyist to enterprise developer nearly frictionless.
Waiting Two Years To Monetize 47:00
Despite explosive early adoption, Ollama had no revenue and no clear business model for roughly two years after Llama 2 took off, a position similar to Docker's own history. The team saw two paths to monetization that could work well for the company, developers, and customers alike: a privacy-focused AI product, which was Ollama's original open-source incarnation, and eventually capturing demand once open models became genuinely capable enough for the hardest problems, such as tool-calling, which closed models handled first. That moment arrived this year with coding agents, where the largest wave of AI consumption finally matched what open models could deliver, confirming the team's instinct to wait for the market rather than force monetization too early.
The Risk Of Treating Users As A Blob 49:01
Building an open-source project that takes off carries a hidden danger, according to the speaker. It becomes easy to think of your user base as an anonymous mass on the internet rather than real people with names and needs. That mindset is risky because it stops you from meeting customers, learning what they are doing, and understanding what they will want six months from now. The speaker admits this is something the team wishes it had done more of over the past few years, and says they are now investing heavily in staying close to their users.
Why They Chose YC The Second Time 49:59
Even as second-time founders, the decision to join Y Combinator was not obvious, and the speaker says they went back and forth on it more than they should have. Starting a company is lonely, even with a strong co-founder, and the speaker's own co-founder, Michael, had been his college roommate at the University of Waterloo. Talking weekly with Jared and a handful of other founder groups during the pandemic eased that loneliness. Moving down and joining the in-person network afterward was even more valuable, connecting them with other founders building on open models and AI. The speaker also credits YC's culture of transparency, noting he still talks with the founder of Docker, now an investor in the company, about mistakes from the prior infrastructure generation that don't need repeating.
Seeing What Good Looks Like 51:31
Having worked on a team that shipped technology to real users with clear product-market fit, even briefly, teaches lessons that are hard to get any other way, the speaker says. Knowing what a healthy bug database looks like, how releases work, and what quality bar a team holds itself to multiplies the odds of later success. This kind of muscle memory, drawn from veterans of Docker and VMware on the team, also shapes accountability: software released today may still be running two years from now, and someone has to answer for how it holds up.
Old Infrastructure Rules Don't Apply 53:32
Many assumptions from the prior DevOps era no longer hold in AI. Platform-as-a-service companies like Heroku, and Docker itself at first, worried about being a vulnerable layer on top of someone else's infrastructure, but in AI, sitting higher in the stack can mean being closer to the customer, which is an advantage. Another broken assumption is the demand for perfect determinism; language models are never perfect, and that unpredictability is treated as a feature rather than a flaw. Staffing has changed too, since AI reduces the need for heavy teams in areas like customer support that once required large headcounts.
Curation As The Real Opportunity 55:30
With a fragmented landscape of models, inference technology, cloud services, and harnesses, the real opportunity lies in curation, making that fragmented universe simply work for developers who just want to build their product. Services like OpenRouter let developers access many providers through one signup and one payment point, while a harness like Open Code lets one interface work across any model, making it easy for developers to test new models against their use case. The speaker frames this as turning today's abundance of models and providers into something usable, closing the conversation by thanking the hosts.
AI-generated summary. It can be wrong or incomplete - check anything that matters against the original.
