Hook
More breakout videos from this creator.
Is Anthropic compute constrained? So yesterday on Instagram Live someone asked me if Anthropic is compute constrained and I said yeah I think so but I was doing two things at once and it was a bit of a distracted answer so I wanted to come back to it because I think what's actually going on is really interesting. Back in February Dario Amodei did two hours with Dwakesh Patel on his podcast and in that he laid out a toy model for how the economics of an AI company works. So you take all the compute you've bought and roughly half goes into training, building the next model and running the research, while the other half goes into inference that's serving you when you use Claude. The inference side makes the money, while the training side makes the capability for the future. He called it research, and you need both. So what error is Anthropic prepared to live with in a compute-constrained world? They're buying data centers a year in advance. If they overestimate how many customers show up, they've got all this inference capacity sitting. And if they underestimate, they're profitable, but they've starved their research budget. Dario called this a hellish demand prediction problem and he's absolutely right, because the time frames, right? So Dwakesh pushed back on him, and he said, look, if you really believe that AGI is two years away, why aren't you spending everything? Go 75-25 in research, take the losses, raise more capital. And Dario's answer was that the research compute has a log linear return. After about $50 billion a year, doubling your spend doesn't double your progress. Okay, so that's the theory. What's happening in practice recently? Anthropic has Mythos. It scores like 15 points higher than Opus on coding benchmarks. I mean, it's pretty amazing. And Anthropic's own language is that it's very expensive to serve. So only a handful of organizations have access through Project Glasswing. You and I don't. That's the 50-50 constraint made really real and tangible. The model exists, but they can't afford to give it to us at any price that makes any kind of sense. Now add to this that remember how OpenAI killed Sora, the AI video tool, and it was burning a million dollars a day in compute with half, fewer than half a million people using it. So they shut it down to free up chips for coding tools and enterprise products. So a company looked at a fixed pool of compute and decides we can't spend it on this anymore. So I think something's changing here. For two years, the story has been AI can do everything. Compute scarcity is now forcing a particular question: What is this stuff actually for? Now, look at where the compute goes right now. A huge amount of it powers things like customer service bots, content generation, coding generation, "doer" work a lot of the time that matters, but it's not necessarily that high value. This is the disconnect that we all know between our individual sense of productivity and what companies say about ROI and what the research says about what people are doing with it and the capabilities that could genuinely matter: finding these critical security flaws, doing real multi-step reasoning in science, those are the ones that are too expensive to run at scale. So Dario's whole thesis is that we're near the end of the exponential for raw pre-training. The gains now come from RL, reinforcement learning, and inference time compute, letting models think longer on hard problems. Both of those cost more per query. The smarter the model gets, the more expensive each question becomes. So the ceiling on what you experience as a user isn't just what AI can do, it's what's worth spending the compute on. Right now, I think what's interesting is the labs are starting to make those choices and companies aren't. They're doing this based on how they see the chips falling, pun intended. Is Anthropic compute constrained? Yeah, sure. So is everyone else. The interesting part is watching what's getting prioritized by whom and what's getting killed. So I'm Helen from Artificiality. I'm going to be doing more lives so what would be helpful to me is if you can put in the comment what time zone works for you. And if you can, do it in Pacific time so I can just look at it at a glance.