How Much Does An AI Datacenter Cost

@technooptimist

17.6K followers · usually 1.1K views a post

Views

143.8K

Likes

2.6K

Comments

151

592

SoundTikTok sound · original sound - technooptimist

Why it worked

The video addresses a high-interest topic (cost of AI infrastructure) with a provocative hook and then breaks down complex financial and technological aspects using clear visuals and expert commentary, making it informative and engaging for a business-minded audience.

Summary

The video discusses the immense cost of building AI datacenters, estimating a 1-gigawatt datacenter to cost around $50 billion. It highlights Nvidia's dominance in training chips but notes the growing competition in inference chips from companies like Groq, Cerebras, AMD, Huawei, Google, and Amazon, suggesting potential challenges for Nvidia in the inference market.

Structure

1Cost of AI datacenters ($50B per gigawatt)
2Elon Musk's projection of scaling to 10 gigawatts ($500B investment)
3Nvidia's dominance in training chips
4Competition in inference chips (Groq, Cerebras, AMD, Huawei, Google, Amazon)
5Nvidia's potential vulnerability in the inference market

Product placement

The video discusses Nvidia's role in AI datacenters and their chips, mentioning competitors like Groq, Cerebras, AMD, Huawei, Google (TPUs), and Amazon (Training chips).

On-screen text

How Much Does An AI Datacenter Cost $50 BILLION THE LAND POWERSHELL IT'S ABOUT $50 BILLION PER GIGAWATT OpenAI AND I THINK THAT'S RIGHT THEY'RE TALKING ABOUT SCALING TO 10 GIGAWATT DATA CENTERS THAT'S A $500 BILLION INVESTMENT I THINK THAT'S RIGHT PER DATA CENTER IF IT GETS THAT BIG TO BE CLEAR THEY'RE STILL WORKING ON THE 1 GIGAWATT DATA CENTERS FROM THERE 2 OR 3 GIGAWATTS NEXT YEAR AND THEN FROM THERE PROBABLY TO 5 NEXT COUPLE OF YEARS SO YOU COULD GET TO A 10 GIGAWATT IN I DON'T KNOW PROBABLY 5 YEARS WHEN I THINK ABOUT NVIDIA'S MARKET THE BIG QUESTION TO ME IS I SO I WOULD BET THE OVER ON DEMAND BECAUSE THAT TO UNDERWRITE DEMAND FOR APPLICATIONS THAT DON'T EXIST YET BUT I'M CONFIDENT I THINK EVERYONE UNDERSTANDS THAT NVIDIA HAS BY FAR THE BEST TRAINING CHIPS BUT THERE'S A LOT OF INCLUDING YOUR COMPANY GROQ THERE'S CEREBRAS AND OBVIOUSLY AMD WANTS TO COMPETE AND THEN HUAWEI IS IS KIND OF A WILD CARD THERE WE CAN TALK ABOUT THAT I SHOULD MENTION ALL THE A6 THAT THE BIG AI COMPANIES ARE MADE UP YEAH USING BROADCOM IP YOU'VE GOT THE GOOGLE TPU'S AND YOU MENTIONED TRAINING WHICH IS AMAZON'S CHIP AND I THINK OPENAI WANTS TO DEVELOP THEIR OWN CHIP SO I THINK THE QUESTION IS GIVEN THAT INFERENCE WILL BE 99% OF THE MARKET AND TRAINING IS ONLY 1% COULD NVIDIA BE IN A SITUATION WHERE THEY'VE GOT TRAINING LOCKED UP BUT WHEN IT COMES TO INFERENCE THERE'S JUST A LOT OF OTHER ALTERNATIVES BECAUSE PEOPLE WILL DEVELOP I'M NOT SAYING THAT'S GONNA HAPPEN THAT WOULD BE THE BIG QUESTION. David Sacks Entrepreneur

More from @technooptimist

More breakout videos from this creator.

Transcript

00:00 gigawatt datacenter 00:01 is about $50 billion 00:02 of investment 00:03 roughly between 00:04 the chips, the land, powershell 00:05 it's about $50 billion per gigawatt 00:06 roughly between 00:07 the chips, the land, powershell 00:08 it's about $50 billion per gigawatt 00:09 it's about $50 billion per gigawatt 00:10 per gigawatt. 00:11 And you have 00:12 you know, Open AI 00:13 and I think that's right 00:14 they're 00:15 talking about 00:16 scaling to 10 gigawatt 00:17 data centers. 00:18 That's a $500 billion investment 00:19 that's a $500 billion investment 00:20 I think that's right 00:21 per data center 00:22 if it gets that big. 00:23 To be clear, 00:24 they're still working on the 1 gigawatt data centers 00:25 from there 00:26 from there 00:27 2 or 3 gigawatts 00:28 next year 00:29 and then from there 00:30 probably to 5 00:31 over the next couple of years. 00:32 So you could get to a 10 gigawatt 00:33 in 00:34 I don't know, 00:35 probably 5 years, something like that. 00:36 When I think about Nvidia's market, 00:37 the big question to me 00:38 is I so 00:39 I would bet the over on demand 00:40 because 00:41 that 00:42 to underwrite demand 00:43 for applications 00:44 that don't exist yet. 00:45 But I'm confident 00:46 I think everyone understands that 00:47 Nvidia has by far 00:48 the best training chips 00:49 but 00:50 there's a lot of 00:51 including 00:52 your company Groq 00:53 and there's Cerebras 00:54 and 00:55 obviously AMD wants to compete 00:56 and then 00:57 obviously Huawei is 00:58 is kind of a wild card there. 00:59 We can talk about that. 01:00 And then 01:01 obviously AMD wants to compete 01:02 and then 01:03 obviously Huawei is 01:04 is kind of a wild card there. 01:05 We can talk about that. 01:06 I should mention 01:07 all the 01:08 A6 01:09 that the big AI companies are made up 01:10 Yeah 01:11 using Broadcom IP 01:12 you've got the 01:13 Google TPUs and 01:14 you mentioned training 01:15 which is Amazon's chip 01:16 and I think OpenAI 01:17 wants to develop their own 01:18 chip. 01:19 So I think the question is 01:20 given that 01:21 inference will be 99% of the market 01:22 and training is only 1% 01:23 could Nvidia be in a situation 01:24 where they've got 01:25 training locked up 01:26 but when it comes to inference 01:27 there's just a lot of other alternatives 01:28 because people will develop 01:29 I'm not saying 01:30 that's gonna happen. 01:31 That would be the big question. 01:32 To me, that would be the big question.