Jalapeño 首次结果展示 AI 推理领域领先的速度与效率

自宣布推出 Jalapeño——OpenAI 首款定制推理芯片以来，我们一直在测试这款芯片及围绕它构建的系统。结果显示，性能取得了显著提升：Jalapeño 能够以单位功耗处理更多 AI 任务，同时更快地返回响应。Jalapeño 通过一种架构同时实现了更高的吞吐量和更低的延迟，而现有硬件系统往往必须在两者之间做出权衡。Since announcing Jalapeño, OpenAI’s first custom inference chip, we have been testing the chip and the system built around it. The results show a significant performance advance: Jalapeño can serve more AI work per unit of power while also returning responses more quickly. Jalapeño delivers both higher throughput and lower latency with one architecture, where existing hardware systems often have to make a tradeoff between the two.

对于客户而言，这意味着更快的响应速度、更灵敏的代理，以及随着需求增长而更加可靠的访问。我们的使命是确保通用人工智能惠及全人类。这些进步将有助于让能力日益增强的 AI 更加经济实惠，并得到更广泛的应用。For customers, that can mean faster responses, more responsive agents, and more reliable access as demand grows. Our mission is to ensure that artificial general intelligence benefits all of humanity. These gains will help make increasingly capable AI more affordable and more broadly available.

OpenAI 模型也加速了 Jalapeño 的开发。较早一代的模型帮助团队设计并启动了这款芯片，而我们最新的模型则在加速其优化和编程方式。Jalapeño 的性能覆盖 GPT‑OSS 120B、DeepSeek R1 和 Kimi K2.5 1T，表明该架构能够适用于在 OpenAI 内部和外部开发的模型。在这三种模型上，Jalapeño 在峰值吞吐量下每瓦所完成的 AI 工作量提高了 1.5 到 1.9 倍，端到端延迟则比对比系统低 1.7 到 3.6 倍。对于高度交互式的工作负载，其性能提高了 2.1 到 4.1 倍。OpenAI models also accelerated Jalapeño’s development. Earlier generations helped the team design and bring up the chip, while our latest models are accelerating how we optimize and program it. Jalapeño’s performance extends across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T, showing that the architecture works across models developed both inside and outside OpenAI. Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. For highly interactive workloads, it delivered 2.1 to 4.1 times higher performance.

Jalapeño 也证明了更广泛的全栈优势。OpenAI 能够协同设计模型、产品、服务软件、芯片、内存、网络和系统，并利用从实际工作负载中获得的经验改进技术栈的每一层。Jalapeño 是一款拥有实测结果的自研芯片，也是一个跨越多代的平台的起点。未来几个月，我们将扩大 Jalapeño 的部署规模，为客户提供更快、更强大、更高效的产品。Jalapeño is also evidence of a broader full-stack advantage. OpenAI can design models, products, serving software, chips, memory, networking, and systems together, using what we learn from real workloads to improve every layer of the stack. Jalapeño is working first-party silicon with measured results, and it is the beginning of a multigenerational platform. In the months ahead, we will ramp Jalapeño to deliver faster, more capable, and more efficient products for our customers.

我们在用户体验相当的条件下评估性能，衡量每个系统在满足客户和交互式智能体所需延迟的同时，单位功耗能够完成多少有用的 AI 工作。这一点对智能体尤为重要，因为它们需要按顺序完成许多步骤，延迟可能会在整个任务中不断累积。We evaluate performance at a matched user experience, measuring how much useful AI work each system can complete per unit of power while meeting the latency customers and interactive agents require. This matters especially for agents, which need to complete many steps in sequence, so delays can compound across an entire task.

为了了解 Jalapeño 在实际应用中的表现，我们在 InferenceX 上对其进行了测试。InferenceX 是 SemiAnalysis 发布的一项公开基准测试，用于衡量服务一项 AI 请求的完整流程。我们在测试涵盖的运行范围内，将 Jalapeño 与领先的商用 AI 系统进行了比较，范围从高吞吐量服务到高度交互、低延迟的应用场景。Jalapeño 在吞吐量、能效和延迟方面实现了更好的综合表现。虽然性能有时会按芯片来报告，但我们认为，单位功耗性能是更有用的衡量标准。To understand how Jalapeño performs in practice, we tested it on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. We compared Jalapeño with leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use. Jalapeño delivered a better combination of throughput, power efficiency, and latency. Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power.

在所有三款公开型号中，Jalapeño 在测试的运行范围内都实现了更优的每瓦性能与延迟组合，因此位于帕累托前沿。为了一致地比较这些系统，我们根据各加速器公布的芯片功耗额定值对结果进行了归一化。Jalapeño 的额定功耗为 700 瓦，不过在所测试的工作负载下，其实测持续功耗始终不超过 550 瓦。Across all three public models, Jalapeño delivered a better combination of performance per watt and latency across the tested operating range, placing it on the Pareto frontier. To compare the systems consistently, we normalized the results using each accelerator’s published chip power rating. Jalapeño is rated at 700 watts, although its measured sustained power remained at or below 550 watts on the workloads tested.

Jalapeño 在 GPT‑OSS 120B、DeepSeek R1 670B 和 Kimi K2.5 1T 上均表现出色。在我们测试过的最大公开模型 Kimi 上，与对比系统相比，它的峰值每瓦性能约高出 1.5 倍，端到端延迟低 3.4 倍。在我们的内部测试中，Jalapeño 在 OpenAI 前沿模型上的优势进一步扩大，这表明随着工作负载规模增大、要求提高，这种架构的价值也会更加突出。Jalapeño performed strongly across GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. On Kimi, the largest public model we tested, it delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. In our internal testing, Jalapeño’s advantage widened further on frontier OpenAI models, suggesting that the architecture becomes more valuable as workloads grow larger and more demanding.

Jalapeño 从一开始的设计理念就是：如果硬件的首要任务是运行现代及未来的语言模型，尤其是交互式智能体，我们会打造怎样的硬件？Jalapeño 的性能提升，源于围绕真实的语言模型工作负载，协同设计芯片、内存、网络、软件和机架级系统。语言模型推理分为几个瓶颈各不相同的阶段。预填充阶段处理提示词，计算密集；解码阶段逐个生成响应词元，更多受内存带宽限制。数据在核心和芯片之间传输时，通信也可能增加延迟，使部分处理单元在等待期间闲置。某个系统即使在一个阶段表现出色，也可能在等待数据或在不同资源之间转移模型状态时失去这一优势。Jalapeño was designed from the start by asking: what hardware would we build if its primary job were serving modern and future language models, especially interactive agents? Jalapeño’s gains come from designing the chip, memory, network, software, and rack-scale system together around real language-model workloads. Language-model inference moves through several distinct phases with different bottlenecks. Prefill, when the system processes a prompt, is compute-intensive, while decode, when the system generates the response token by token, is constrained more by memory bandwidth. Communication can also add latency when data must move between cores and chips, leaving some processing units idle while they wait. A system that excels at one phase can lose that advantage while waiting for data or moving model state between different resources.

我们设计 Jalapeño 时，致力于最大限度减少数据搬运和通信延迟。这意味着，可以明确放置模型状态（包括生成响应时使用的 KV 缓存），并将其保留在本地；与此同时，系统会针对每个推理阶段启用恰当组合的计算、内存和网络资源。网络是这一架构不可或缺的一部分。其覆盖范围广，使整个工作负载能够始终运行在同一个互联系统内，从而减少数据搬运，并帮助完整请求从头到尾保持快速、高效。最终，我们打造出一种均衡且灵活可调配的加速器，能够支持不断变化的模型架构，在预填充和解码阶段均表现出色，并随着二者之间的负载平衡变化而灵活适应——这正是智能体工作负载的一项显著特征。We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase. The network is integral to the architecture. Its large domain allows the entire workload to remain within one connected system, minimizing data movement and helping the complete request stay fast and efficient from beginning to end. The result is a balanced and fungible accelerator that can support changing model architectures, excel at both prefill and decode, and adapt as the balance between them changes, a defining feature of agentic workloads.

AI 在 Jalapeño 的开发过程中发挥了直接作用：团队通过探索不同实现方案、缩短设计、测量和验证周期，并持续迭代模型工作负载，在九个月内完成了从初步设计到流片的过程。AI 还帮助优化了芯片的算术电路，使团队得以按计划在芯片中集成更强的计算性能。AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule.

Jalapeño 被设计成一种清晰、可预测的编程目标，供人类和 AI 使用。工程师可以通过本地张量、显式通信和可预测的同步来描述工作。随后，AI 可以优化这些工作的映射、放置、调度，以及在整个系统中的协调方式。这种清晰、可预测的结构，为 AI 提供了一种切实可行的方法，来应对传统上颇具挑战性的并行编程问题。Jalapeño was designed as a clear, predictable programming target for both humans and AI. Engineers can describe work through local tensors, explicit communication, and predictable synchronization. AI can then optimize how that work is mapped, placed, scheduled, and coordinated across the system. That clear, predictable structure gives AI a tractable way to tackle the traditionally difficult problem of parallel programming.

支持每个新模型系列仍然需要开发新的内核并进行针对该模型的优化。团队使用 Codex 和 GPT‑Astra，在两个月内让三个未纳入 Jalapeño 原始生产计划的开放权重模型实现了高性能。这既展现了该架构的灵活性，也体现了 AI 协助编程的速度。对于 GPT‑OSS 中选定的注意力模块和专家混合模块，AI 生成的实现比现有的人类专家编写的实现快 1.5 到 1.8 倍。这些数据仅适用于选定模块，并不代表整个模型，但它们预示着一种强大的新开发循环。Supporting each new model family still requires new kernels and model-specific optimizations. Using Codex with GPT‑Astra, the team brought three open-weight models that were not part of Jalapeño’s original production plan to high performance within two months. This demonstrated both the flexibility of the architecture and the speed at which AI can help us program it. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations. Those figures apply to the selected blocks, not the full model, but they point toward a powerful new development loop.

AI 基础设施的价值在于它能够助力完成有用的现实工作。通过利用相同的电力和硬件产出更多有用成果，Jalapeño 可以帮助我们满足更多需求，并降低成功交付成果的成本。对 OpenAI 而言，这有助于提升运营杠杆效应，使有用工作和收入的增长速度超过服务成本。它还可以推动更广泛的应用，并支持持续投资于更好的模型、产品和基础设施。更快的推理能够加速迭代，并催生新的用例。AI infrastructure is valuable because of the useful real-world work it enables. By producing more useful work from the same power and hardware, Jalapeño can help us serve more demand and lower the cost of delivering a successful result. For OpenAI, that can improve operating leverage by allowing useful work and revenue to grow faster than the cost to serve. It can also support broader adoption and continued investment in better models, products, and infrastructure. Faster inference can enable faster iteration and new use cases.

Jalapeño 拓展了高效、低延迟推理的可能性：Jalapeño expands what is possible for efficient, low-latency inference:

• 以此前只有快速模式才能实现的效率进行超快速模式推理
  Ultra-fast-mode inference at efficiencies previously available only in fast mode
• 快速模式推理，效率达到此前只有批处理模式才能实现的水平
  Fast-mode inference at efficiencies previously available only in batched mode
• 提高批处理模式推理的效率
  Higher efficiency for batched-mode inference

我们计划在今年年底前开始在 OpenAI 的计算基础设施中部署 Jalapeño。它是一个跨越多代的路线图中的第一代：第二代正在深入开发，第三代也已初具雏形。每一代都会借鉴我们积累的经验，进一步提升效率和速度。We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed.

满足日益增长的 AI 需求，需要从所有可用来源获取更多算力。我们将继续广泛部署 NVIDIA 及其他合作伙伴提供的加速器，用于训练和推理工作负载。我们的使命是确保通用人工智能造福全人类。Meeting growing demand for AI will require more compute from every available source. We will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads. Our mission is to ensure that artificial general intelligence benefits all of humanity.

随着我们为部署做准备，我们将继续进行生产资格认证、完善软件、为大规模运行 Jalapeño 做准备，并在更多模型上验证性能。迄今为止的成果表明，当我们协同设计整个系统时，能够实现怎样的可能：以更高的效率，让更多人用上响应更迅速、能力更强、智能体特性更突出的 AI。As we prepare for deployment, we are continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models. The results so far show what is possible when we design the full system together: more responsive, capable, and agentic AI delivered more efficiently to more people.