OpenAI says its new chips can outperform Nvidia processors

@bloombergbusiness

1.1M followers · usually 7.6K views a post

Views

120.6K

Likes

3.2K

Comments

246

Shares

142

SoundTikTok sound · original sound - bloombergbusiness

Promoted product

Jalapeno

Why it worked

The video likely performed well due to its focus on a highly anticipated and competitive area in AI hardware development, directly comparing a new product from a major player (OpenAI) against an established leader (Nvidia). The clear demonstration and explanation of the chip's technical advantages, particularly its low latency and efficiency, would appeal to a tech-savvy audience interested in the future of AI.

Summary

The video discusses OpenAI's new 'Jalapeno' AI chips, highlighting their performance advantages over Nvidia processors in terms of throughput and ultra-low latency. The head of hardware at OpenAI explains the chip's design and architecture, emphasizing its HBM4 memory and how it reduces data movement to optimize AI workloads.

Structure

1Introduction of OpenAI's new chip, 'Jalapeno'
2Comparison with Nvidia processors, focusing on performance metrics
3Explanation of the chip's design, emphasizing HBM4 and low latency
4Discussion of the 'blank slate' design approach and its benefits

Original caption

OpenAI says it new Jalapeno chips performed better than Nvidia’s current lineup during testing, leading in two categories: the amount of AI work it could handle per unit of power and its speed at returning responses. OpenAI Head of Hardware Richard Ho goes over the results. #OpenAI #Nvidia #tech #business

More from @bloombergbusiness

Product placement

OpenAI (company): ACTS - develops and sells AI chips. Removing it would change the video's topic. Nvidia (company): ACTS - competitor to OpenAI's chips. Removing it would change the video's topic. Jalapeno (AI chip): ACTS - the specific AI chip being discussed and shown. Removing it would change the video's topic. Bloomberg News (company): APPEARS - identified as the source of the report via on-screen text and creator handle. Removing it would not change the video's topic. HBM (memory technology): ACTS - a type of memory used in the chip. Removing it would change the video's topic. SRAM (memory technology): ACTS - a type of memory used in the chip. Removing it would change the video's topic. HBM4 (memory technology): ACTS - a specific version of HBM memory used in the chip. Removing it would change the video's topic. GPUs (graphics processing units): ACTS - a type of processor that OpenAI's chip is being compared to. Removing it would change the video's topic. TPUs (tensor processing units): ACTS - a type of processor that OpenAI's chip is being compared to. Removing it would change the video's topic.

On-screen text

OpenAI says its new chips can outperform Nvidia processors This is it. This is Jalapeno. This is Jalapeno, you very graciously allowed me to get my hands on it. Ed Ludlow Bloomberg News allowed me to get my hands on it. And again, the whole emphasis is on throughput and ultra-low latency. And actually, in the ultra-low latency piece, you'd associate that with SRAM based systems. This is both HBM and SRAM. Talk a little bit about its design and architecture, but why that ultra-low latency matters? So, it is a HBM-based design, it's HBM4. Richard Ho OpenAI Head of Hardware It's one of the first, you know, processing to go to volume with HBM4, and we're right behind Nvidia's products, and it is very low-latency. The reason how we got it was really taking a blank slate approach to this design. So my team arrived at OpenAI, and we basically were able to work with the research team to understand where the bottlenecks were in large language models. And with a blank slate design, we figured out that you had to reduce your data movement a lot, and we figured out a way to do it. And so this architecture is different from GPUs, is different from TPUs and other chips in that nature. It really reduces the data movement, it optimizes the algorithm, and then it basically is able to get this very-low latency, which matters for the agentic workloads.

Transcript

This is it. This is Jalapeno. This is Jalapeno, you very graciously allowed me to get my hands on it. Ed Ludlow Bloomberg News allowed me to get my hands on it. And again, the whole emphasis is on throughput and ultra-low latency. And actually, in the ultra-low latency piece, you'd associate that with SRAM based systems. This is both HBM and SRAM. Talk a little bit about its design and architecture, but why that ultra-low latency matters? So, it is a HBM-based design, it's HBM4. Richard Ho OpenAI Head of Hardware It's one of the first, you know, processing to go to volume with HBM4, and we're right behind Nvidia's products, and it is very low-latency. The reason how we got it was really taking a blank slate approach to this design. So my team arrived at OpenAI, and we basically were able to work with the research team to understand where the bottlenecks were in large language models. And with a blank slate design, we figured out that you had to reduce your data movement a lot, and we figured out a way to do it. And so this architecture is different from GPUs, is different from TPUs and other chips in that nature. It really reduces the data movement, it optimizes the algorithm, and then it basically is able to get this very-low latency, which matters for the agentic workloads.