OpenAI unveils custom AI chip Jalapeño with HBM4 memory, threatening Nvidia's inference market

• OpenAI's first custom AI chip, Jalapeño, is expected to begin deployment in the company's computing infrastructure by the end of the year.
• Analysts told CNBC that it could put pressure on Nvidia in the fast-growing inference market and reduce OpenAI's reliance on Nvidia for some workloads.
• Jalapeño beat Nvidia's Blackwell systems on performance per watt in nearly all tested scenarios, according to OpenAI, but analysts said Nvidia's newer platform is a more like-for-like comparison.

Nvidia's near-monopoly over the most advanced AI chips is under "threat" as OpenAI and other tech giants announce custom-built semiconductors, analysts told CNBC.

OpenAI announced that its first AI chip, the Jalapeño, had "industry-leading speed and efficiency," as it unveiled the semiconductor on Tuesday. Google, AWS and Meta are all also developing their own AI chips.

Nvidia has seen its share price rocket amid the data center buildout, which has created huge demand for its chips in both model training and inference — how AI systems run day-to-day tasks.

But hyperscalers and AI companies are increasingly gaining ground in developing their own silicon to power AI systems.

The Jalapeño chip, designed for inference, shows that a "hyperscaler-designed chip can now match or beat Nvidia's Blackwell-class GPUs on inference efficiency," Adrien Sanchez, technology analyst at Yole Group, told CNBC.

He added that, while Nvidia still owns the "vast majority" of AI compute and has ecosystem lock-in to its software platform CUDA, OpenAI's new chip is a "threat to Nvidia's inference margins, which is the field growing the most at the moment."

Nvidia has been approached for comment.

OpenAI announced the first benchmarking results from its Jalapeño, saying it would allow users to get "faster responses, more responsive agents and more reliable access" as demand grows.

The new chip is being developed with Broadcom. It will be deployed within OpenAI's compute infrastructure by the end of the year, and OpenAI said it was already working on the semiconductor's generations two and three.

Industry-Leading Performance

OpenAI announced Jalapeño in June, saying it would be "built from the ground up for current and future LLMs across the industry."

Jalapeño is an "impressive achievement, most of all in terms of efficiency," Alexander Harrowell, senior principal analyst at Omdia, told CNBC.

"In a large-scale deployment, this would save power, cooling, and power distribution infrastructure, and contribute a lot to their unit economics," he added.

OpenAI's custom chip could reduce its reliance on Nvidia over time for inference workloads, TrendForce Analyst Fion Chiu told CNBC.

But for more compute-intensive workloads, like large-scale model training and frontier AI workloads, "we believe Nvidia GPUs will remain important given their broad programmability, performance, software ecosystem, and ability to handle a wide range of workloads," Chiu said.

How OpenAI's Jalapeño Chip Compares with Nvidia

Research firm SemiAnalysis said it visited OpenAI's labs to benchmark Jalapeño. It found the chip beat Blackwell on performance per watt in nearly all tested scenarios. But it added that the comparison was "somewhat incomplete and unfair" because Jalapeño uses newer HBM4 memory. Nvidia's Rubin platform is a better like-for-like comparison as it also uses HBM4.

"Jalapeño is really competing against chips like Rubin that also use HBM4," SemiAnalysis analysts said in a blog post titled "OpenAI Jalapeño: Better Than Nvidia Blackwell" on Tuesday.

"Vera Rubin systems are starting to ship to customers right now, while it will still be some time before OpenAI has anything beyond engineering samples of Jalapeño," the blog post added.

Nvidia Competitors Gain Ground

OpenAI is one of several AI companies looking to build custom silicon.

In April, a slew of deals were announced for custom AI chips, also known as application-specific integrated circuits (ASICs).

• Google unveiled new chips for AI training and inference, which it calls tensor processing units (TPUs).
• Meta said it agreed to deploy 1 gigawatt of custom AI chips using Broadcom technology as part of a multi-GW deal.
• Anthropic said it committed to spending more than $100 billion on AWS tech over the next 10 years, including current and future generations of Amazon's custom AI chips, Trainium.

"Omdia expects custom ASIC chips like Jalapeño to exceed GPUs in volume by 2028, although revenue will" —