Chips & Hardware
OpenAI's Jalapeño chip beats Nvidia Blackwell on new inference benchmarks
OpenAI shared benchmark results for its in-house inference chip, Jalapeño, showing better tokens-per-user and throughput-per-kilowatt than Nvidia's current state-of-the-art system, with small-volume deployment expected by the end of 2026.
At the Hot Chips conference Tuesday, OpenAI shared a more detailed look at Jalapeño, including the first batch of benchmark results for the new system. Tested on SemiAnalysis’ InferenceX benchmark, Jalapeño registered both more tokens per user and more throughput per kilowatt than currently available state-of-the-art inference processors.
“The bottom line is that the results show a very, very significant performance advance over state of the art,” Richard Ho, OpenAI’s head of hardware, said on a press call. “Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It’s very efficient to serve a lot of customers, but it can also be very low latency.”
The comparison was against an Nvidia Blackwell system, but by the time Jalapeño reaches full deployment, the competition may have advanced significantly. Ho estimated Jalapeño would deploy at the end of 2026 “in very small volumes,” with more significant deployment coming in 2027.
First announced last October, Jalapeño was developed by OpenAI in close collaboration with Broadcom, with OpenAI’s own models assisting in the development process. The company plans to make Jalapeño a multigenerational platform, allowing its AI products, models, chips, and memory to all be developed in concert.
That full-stack approach let OpenAI target specific phases of the inference process that often cause friction, particularly the prefill and communication phases, which the company says often act as bottlenecks. OpenAI said it designed Jalapeño to minimize data movement and communication delays, so that model state — including the KV cache used while generating a response — can be kept local while the system activates the right mix of compute, memory, and networking for each inference phase.
Why it matters
The benchmark is OpenAI’s first public evidence that building its own inference silicon with Broadcom can beat off-the-shelf Nvidia hardware on efficiency — though the real test comes once Jalapeño ships at meaningful volume in 2027.