Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Warm welcome to Basil Chatha , our newest cohost from the Forward Deployed side of the AI Engineering world! We’ll be ramping up a new show with him and couldn’t pass up an interview with Modal’s CEO Eric on the occasion of their monster Series D! Two years ago, we spoke with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, predicting load, or paying for idle resources. CTO Akshat then joined us to talk about Modal’s infra primitives and stack to offer those primitives and agent experience. A lot has changed in the past 5 months. Since their $5B series C, Modal now reports it crossed $1 billion in annualized run rate , raised a $750 million Series D at a $16B valuation , and has spun up nearly three billion sandboxes. In this conversation, Erik joins Vibhu to explain the infrastructure and economics behind that growth—and why demand for compute rarely looks like the contracts used to buy it. Watch the full conversation on YouTube → GPU demand isn’t rectangular A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle. Real workloads behave differently: they grow, spike, and fall throughout the day. Buy enough for the peak and you pay for idle hardware; buy less and you risk missing demand. Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it. The smoother the aggregate demand, the more useful that pool can be. It’s a useful companion to our conversation with SF Compute’s Evan Conrad : Evan focuses on making capacity tradeable; Erik focuses on making it usable in smaller, elastic slices. There is a financial risk on the other side. If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply. Erik says he doesn’t expect an imminent collapse, but thinks the possibility matters when building the business. Jump to the GPU price discussion → The sandbox that almost didn’t make it Modal launched sandboxes after Erik and cofounder Akshat Bubna noticed that language models could generate code—and that someone would need a place to run it. Early adoption was modest enough that the team discussed shutting the product down. Then coding agents, background agents, and reinforcement learning created new demand. Erik says sandboxes have grown to roughly a third of Modal’s revenue, a result he didn’t predict at launch. That growth also changes what infrastructure has to do. An agent may create an environment, run code, inspect the result, and try again many times. The latency of starting and restoring those environments becomes part of the experience. In our recent interview with Akshat , we explored this shift from developer experience to agent experience; here Erik adds the product and business story. Jump to the sandbox discussion → Fast startup is an infrastructure feature Elasticity only works if capacity can arrive quickly enough to matter. Modal’s serverless GPU write-up describes the machinery: a buffer of available GPUs, a custom filesystem, process checkpointing, and GPU memory snapshots. Those pieces reduce the work needed to bring another model replica online. It is a concrete example of how infrastructure decisions show up in an application’s responsiveness and cost. We also discuss Erik’s path from Spotify’s recommendation system to Modal, how the company buys compute, alternatives to NVIDIA, forward-deployed engineers, and why code review is optional at Modal. Jump to engineering culture → Show notes Modal — the AI infrastructure platform Erik founded. Erik’s writing — his essays on infrastructure, markets, and building companies. Modal’s Series D announcement — the company’s latest reported growth and sandbox milestones. How Modal achieved truly serverless GPUs — the startup and scaling techniques behind elastic inference. Modal’s resource solver — how it selects capacity across changing prices and availability. Scaling to one million concurrent sandboxes — an engineering account of sandbox scaling. Our first interview with Erik — the original serverless infrastructure thesis. Our interview with Akshat Bubna — infrastructure and developer experience for agents. The Programmable Data Center — Evan Conrad on GPU trading and physical settlement. Timestamps 0:00 What Modal does 1:25 From Spotify to building Modal 4:19 Founding before the generative AI boom 7:43 First customers and fundraising 10:00 Building for inference and autoscaling 13:21 Where Modal gets GPU capacity 15:39 Renting vs. owning infrastructure 19:45 The risk of GPU prices falling 24:18 Is AI demand infinite? 26:12 Alternatives to NVIDIA 28:25 What Modal can learn from neoclouds 30:26 GPUs as an asset class 31:55 Modal’s products: functions and sandboxes 36:03 The case for smaller training workloads 41:21 How agent sandboxes are changing 46:48 Anthropic and sandbox security 49:29 Using agents to test security 52:51 Developer experience in the age of agents 57:45 What Modal’s forward-deployed engineers do 59:55 Modal’s approach to product management 1:03:29 Optional code reviews and engineering culture 1:05:56 Cloud, on-prem and bring-your-own-cloud 1:09:25 Capacity, fundraising and growth 1:13:07 Scaling the team 1:17:28 Erik’s tweets: ephemeral sandboxes 1:18:31 The problem with rectangular GPU contracts 1:21:20 Inference providers vs. model builders 1:23:25 Where models improve next Transcript Automatically transcribed from the revised edit; names and wording may contain errors. 0:00 Thanks for coming on, Eric Fern-Hartson from Modal. Could you explain the business a little bit and maybe how you got here? Yeah. So we build, the easiest way to put it is we build AI infrastructure. So we offer a compute platform and a storage platform that enables a lot of customers like Dacagon, and Suno, and Runway, and Cognition, and many others to essentially build AI applications. So what that means in practice is a lot of inference, a lot of sandboxes, sometimes training and fine-tuning, sometimes batch jobs and other things. So we have basically built a software SDK that enables customers to move really fast and build this. We manage all the capacity underneath, make it easier to deploy and scale and run these things in production. So you were on Light and Space like two years ago, a lot has happened since then. How big were you back then, like whatever numbers you can share and comparatively, how big are you now? I actually don’t remember where we were, but it’s probably like at least 50X, there’s revenue and users and users. Yeah. I use it actually for a side project that I have. Nice. Yeah. It’s like, yeah, turn 2D images into 3D models. Yeah. So yeah, just hosting a couple of models here. What do you think? Any feedback? I mean, I love it. Nice. Good. I mean, yeah, that’s why I use it over any competitors or anything. Good. So before you started Modal, you were at Spotify and also like better.com, I think. 1:30 That’s right. Yeah. Yeah. What were you doing at Spotify? I actually did a lot of different things at Spotify, but my main contribution was I built a music recommendation system. The first version I joined after school in 2008, 2009 maybe, and spent seven years at Spotify. And most of that time was building basically from scratch and music recommendation system and built up a small team and we launched a bunch of features like related artists and Discover Weekly and a bunch of other stuff. So a lot of, a lot of matrix factorization, a lot of compute intensive stuff. And arguably like a lot of what sort of later became the genesis of Modal was just building AI applications and realizing like the infrastructure fucking sucks. Like, there’s got to be a better way to do that. So a lot of sort of, you know, the rationale f