Launch Context: At the Hot Chips 2026 conference, OpenAI, in collaboration with Broadcom, unveiled “Jalapeño,” a proprietary ASIC dedicated to inference. It is not a consumer product but is designed exclusively for OpenAI’s own data centers, with large-scale deployment scheduled to begin in late 2026. This review analyzes this highly anticipated AI computing chip across multiple dimensions—including hardware architecture, measured performance, strengths and weaknesses, and industry impact—based on official documentation and third-party benchmark testing (SemiAnalysis InferenceX).
I. Product Positioning: A Specialized AI Processor Built Solely for Inference
From its inception, Jalapeño eschewed the general-purpose route; it does not support large model training, dedicating all hardware transistor resources to large model inference.
Historically, the industry standard relied on general-purpose GPUs to handle both training and inference, a compromise that sacrificed inference energy efficiency for versatility. In contrast, faced with the massive concurrent inference workloads of ChatGPT and AI agents, OpenAI identified its core challenges not as theoretical raw compute power, but as three critical online service metrics: token throughput, end-to-end latency, and output per unit of power consumption.
- Development Model: OpenAI defined the architecture; Broadcom handled silicon implementation and I/O links; Celestica managed board and system integration; and TSMC manufactured the chips using the N3P process.
- Development Cycle: The timeline from architectural design to tape-out was just nine months—significantly shorter than the industry-standard 18–36 months for ASICs. OpenAI accelerated this process by utilizing its own large models to assist in chip design verification.
- Key Hardware Specifications: Die area of 840mm² (approaching the reticle limit for EUV lithography); package TDP of 700W (with measured sustained operational power of ~550W); equipped with HBM4 memory delivering 15.4TB/s bandwidth; FP8 as the core operational precision, featuring deep hardware optimizations tailored for LLM decoding and KV Cache operations. – Business Model: Not sold externally; cloud instances are not available to the public. The entire supply is dedicated to OpenAI’s own operations; general users can only indirectly benefit from its computing power via ChatGPT and the OpenAI API.
II. Architectural Analysis: Targeting the True Bottlenecks of Inference and Minimizing Inefficient Data Movement
Large model inference consists of two stages: Prefill (input understanding) and Decode (token-by-token output). For the vast majority of production workloads, the bottleneck lies not in the compute units themselves, but in memory bandwidth, KV Cache read/write operations, and the power consumption and latency costs associated with data movement.
Jalapeño employs a highly optimized systolic array architecture that locally pairs compute units with partitioned HBM4 memory. This design keeps weights and KV Cache in nearby storage as much as possible, minimizing data migration across chips and memory modules, and shifting the power budget from data movement to actual computation.
Unlike GPUs, which offer powerful general-purpose programmability, Jalapeño strips away hardware units unnecessary for inference scenarios, concentrating all resources on the large-matrix operations characteristic of Transformer models. Additionally, OpenAI developed a proprietary compiler stack that compiles model operators directly into ASIC hardware—bypassing the CUDA ecosystem—to achieve hardware-software co-optimization.
Core Design Philosophy: Do not waste transistors or power on unused functions; dedicate the entire resource budget to real-world production inference workloads.
III. Benchmark Results: Significant Lead in Performance-per-Watt; Dominance in Low-Latency Scenarios
Testing was conducted using the SemiAnalysis InferenceX benchmark suite. Models included GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T, benchmarked against NVIDIA’s Blackwell series (GB200 and GB300). No software acceleration techniques—such as multi-token prediction—were enabled during testing.
Key Test Results
- Energy Efficiency (Mixed-Token Throughput per Kilowatt)
Jalapeño’s peak energy efficiency is 1.5–1.9 times that of the comparison NVIDIA systems. With the GPT-OSS 120B model, Jalapeño achieves a throughput of 85,448 tokens per kilowatt, compared to 44,960 tokens for the GB200; this allows the same amount of power to handle nearly twice the volume of user requests. For the trillion-parameter Kimi K2.5 model, energy efficiency reaches 1.5 times that of the GB300. - End-to-End Latency and Per-User Generation Speed
The advantages in low-latency interaction scenarios are further amplified: in tests using DeepSeek R1 670B, Jalapeño’s end-to-end latency was 1.65 seconds, versus 5.99 seconds for the GB300. Single-user decoding speed reached 700 tokens/s—compared to 169 tokens/s for the GB300—significantly reducing user wait times.
When maintaining a fixed Quality of Service (QoS) that ensures a user experience of 100 tokens/s, Jalapeño can support several times the number of concurrent users compared to the GB300, offering significant value for high-concurrency services like ChatGPT. - Gains in Agent Scenarios
For Agent workloads involving multi-turn calls and long conversations, official data shows that the combined throughput and latency advantage widens to a factor of 2.1–4.1; these are also core scenarios for OpenAI’s long-term business operations.
Note: These tests were conducted in a controlled laboratory environment using a standard workload of 8k input tokens and 1k output tokens; there is currently no large-scale public test data regarding real-world performance in production clusters involving ultra-long contexts or complex multi-turn Agent interactions.
IV. Key Advantages
- Significant Reduction in Inference Costs
Internal evaluations by OpenAI indicate that, for equivalent workloads, the cost of inference per token on Jalapeño is approximately 50% lower than on existing GPUs. Given OpenAI’s annual inference expenditure of several billion dollars, large-scale deployment could yield massive cost savings while creating room for API price reductions and product scaling. 2. Balancing Throughput and Low Latency
Traditional hardware often faces a trade-off where “high throughput inevitably increases latency.” The Jalapeño architecture optimizes both metrics simultaneously, supporting massive concurrent user traffic while ensuring rapid response times for individual user interactions—perfectly aligning with the needs of consumer-facing large model products. - Establishing a Full-Stack AI Closed Loop
This creates a flywheel effect where “large models participate in chip design, and chips run large models.” By controlling the entire chain—models, compilers, chips, and data centers—OpenAI can continuously iterate its hardware for future GPT series and agent models, rather than passively following the product roadmaps of third-party chip vendors. - Power Efficiency Suited to Data Center Constraints
The primary constraint for AI data centers is often not the number of servers, but the power supply limit. Jalapeño’s higher output per watt means more AI services can be deployed within the same power allocation, alleviating the power bottlenecks associated with computing infrastructure.
V. Objective Shortcomings and Risks: Not an “Nvidia Killer”
While much industry commentary portrays Jalapeño as a disruptive product that crushes Nvidia, the chip itself involves clear trade-offs and risks; it should not be overhyped.
- Complete Abandonment of Training Capabilities; No Total Escape from GPU Dependency
Jalapeño is limited to inference tasks; training for next-generation GPT large models still relies on high-end GPU clusters. The strategy simply brings the inference workload—which accounts for the bulk of daily operational costs—under internal control. - Risks Associated with Specialized ASICs
As a fixed-function ASIC, the hardware logic is hard-wired. If large model architectures undergo major shifts—such as the emergence of entirely new attention mechanisms or sparse computing paradigms—Jalapeño hardware lacks the flexibility of GPUs to adapt. Its energy-efficiency advantage would rapidly diminish, creating a high dependency on OpenAI’s future models remaining compatible with the chip architecture. - Benchmarks Do Not Reflect Large-Scale Production Realities
Current data stems from laboratory sample testing and has not yet been validated through long-term production in large-scale clusters comprising tens of thousands of chips. Factors such as ASIC yield, thermal management, cluster interconnects, and fault tolerance require validation in real-world operational environments. SemiAnalysis merely witnessed the chip running on-site; this was not a comprehensive, fully independent third-party test. 4. Closed Ecosystem: Unavailable for External Use
Like Google’s TPU and Amazon’s Trainium, this is infrastructure developed by a tech giant for internal use; it is not sold commercially, meaning other companies in the industry cannot directly purchase the chip to achieve the same performance benefits.
VI. Industry Implications: The Wave of In-House Chip Development by Model Providers
The significance of Jalapeño lies not in creating a chip with superior benchmark scores, but in demonstrating that leading large-model providers are fully capable of developing their own ASICs, thereby retaining control over the cost, performance, and roadmap of inference computing power.
With Google’s TPU, Amazon’s Trainium, Microsoft’s Maia, Meta’s MTIA, and now OpenAI’s Jalapeño, the world’s leading AI companies are collectively moving toward vertical integration of hardware and software. While general-purpose GPUs remain the top choice for training, general development, and heterogeneous model experimentation, specialized ASICs are capturing an increasing share of the computing market for massive, fixed-workload inference tasks.
For Nvidia, Jalapeño is not a direct competitor, but it will alter customer purchasing logic: in the future, hyperscale AI clients will shift a portion of their inference workloads to in-house hardware, diverting demand away from GPUs. The dimension of market competition has evolved from a simple contest of chip hardware to a rivalry based on full-stack capabilities—spanning models, compilers, chips, and data centers.
VII. Summary and Outlook
Jalapeño is a chip with a highly focused objective; rather than striving for versatility, it concentrates all its capabilities on large-model inference. In laboratory benchmarks, it has delivered impressive results regarding energy efficiency and latency, offering a potential solution to the critical business challenge of inference costs for OpenAI.
However, its ultimate value cannot be definitively assessed until it undergoes large-scale cloud deployment between late 2026 and 2027. If Jalapeño maintains its laboratory performance levels under real-world, massive user workloads, we can expect more AI companies to follow suit by developing their own inference chips; conversely, should it encounter practical issues—such as yield rates, software compatibility, or architectural iteration challenges—the luster of the ASIC approach may dim.
OpenAI has already announced the codename for its second-generation chip, Serrano; Jalapeño represents the starting point, not the end, of OpenAI’s in-house hardware development journey. In a nutshell: Jalapeño proves that AI model companies can successfully build chips, but its deployment is limited to OpenAI’s own data centers; it still has a long way to go before disrupting the broader AI chip market.