NVIDIA Rubin Platform Targets 10x Lower Inference Cost
Six-chip codesign pairs Vera CPUs, Rubin GPUs, and NVLink 6 to cut token spend and scale MoE and agentic AI in 2026 clouds.
Overview
NVIDIA has unveiled the Rubin platform, a next-generation data center stack built to reduce the cost of training and serving large AI models. Announced January 5, 2026 at CES, Rubin combines six new chips into a single rack-scale architecture and pairs that hardware with tightly integrated system software. The announcement also lands amid broader momentum in agentic workflows and recent improvements in NVIDIA's AI agent development tools, reinforcing that hardware and software are moving in tandem.
For developers, the headline claim is economic: NVIDIA says Rubin can deliver up to a 10x reduction in inference cost per generated token compared with the NVIDIA Blackwell platform. For mixture-of-experts (MoE) training, NVIDIA also claims up to 4x fewer GPUs are needed, which can change both cluster sizing and budget planning.
What NVIDIA actually launched
Rubin is positioned as a “one AI supercomputer” platform created through extreme codesign across compute, networking, and infrastructure control:
- NVIDIA Vera CPU
- NVIDIA Rubin GPU
- NVIDIA NVLink 6 Switch
- NVIDIA ConnectX-9 SuperNIC
- NVIDIA BlueField-4 DPU
- NVIDIA Spectrum-6 Ethernet Switch
NVIDIA is also introducing two main system forms:
- Vera Rubin NVL72, a rack-scale system combining 72 Rubin GPUs and 36 Vera CPUs with NVLink 6, ConnectX-9, and BlueField-4
- HGX Rubin NVL8, an 8-GPU board aimed at more conventional server deployments (including x86 platforms)
Key features developers should care about
Lower-cost inference for long-context and reasoning
Rubin targets the shift from single-turn chat to multi-turn, agentic workloads where token counts and context windows grow quickly. NVIDIA attributes the cost reduction to a mix of GPU throughput, interconnect bandwidth, and system-level efficiency.
For teams comparing options for latency and cost per token, it can also be helpful to benchmark against high-performance AI inference platforms such as Groq, especially when you are planning production-grade long-context serving.
As agentic systems get more complex, it is not just throughput that matters. You also need repeatable ways to validate tool use, policy compliance, and reliability, so frameworks for evaluating agentic AI behaviors become part of the practical stack alongside accelerators and interconnect.
NVLink 6 for MoE scale-out
MoE models are communication-heavy because token routing increases all-to-all traffic. NVIDIA’s sixth-generation NVLink provides 3.6 TB/s per GPU, and NVIDIA claims the NVL72 rack reaches 260 TB/s of total bandwidth. Practically, this is aimed at keeping experts fed without paying the latency tax of external networking.
Vera CPU and C2C connectivity
Vera is designed as a CPU companion for AI factories with 88 custom Olympus cores and Armv9.2 compatibility, plus NVLink-C2C connectivity for higher bandwidth CPU to GPU paths. For builders operating large clusters, CPU efficiency matters for orchestration, preprocessing, and data pipeline overhead.
BlueField-4 and AI-native storage for KV cache
Rubin introduces the Inference Context Memory Storage Platform, using BlueField-4 as a storage processor to share and reuse key-value cache across infrastructure. This is tailored to agentic reasoning systems that revisit context repeatedly, potentially lowering recompute and improving throughput predictability.
Confidential Computing at rack scale
NVIDIA says Vera Rubin NVL72 is the first rack-scale platform to provide confidential computing across CPU, GPU, and NVLink domains. For teams serving regulated industries, this can simplify architecture decisions around isolation and model protection.
Networking and reliability upgrades
Rubin expands NVIDIA’s Spectrum-X Ethernet roadmap with Spectrum-6 and Ethernet photonics switch systems. NVIDIA claims these deliver 5x better power efficiency and improved reliability and uptime characteristics, aimed at keeping large AI factories stable under continuous training and inference load.
Impact for developers and platform teams
Rubin’s biggest implication is that inference economics may shift again, especially for:
- MoE inference at scale, where interconnect and cache behavior dominate cost
- Agentic systems, where sharing and persisting context is becoming a first-class performance problem
- Multi-tenant bare-metal clusters, where isolation and control planes are as important as raw FLOPS
If NVIDIA’s token-cost claims hold in real deployments, teams may be able to run higher-quality reasoning models (or longer contexts) within the same spend, and infrastructure teams may revisit GPU count assumptions for MoE training.
Ecosystem and availability timeline
NVIDIA says Rubin is in full production, with partner systems expected in the second half of 2026. Cloud providers slated to offer Vera Rubin-based instances in 2026 include AWS, Google Cloud, Microsoft Azure, and Oracle Cloud Infrastructure, alongside NVIDIA cloud partners such as CoreWeave, Lambda, Nebius, and Nscale.
Microsoft highlighted its “Fairwater” AI superfactory direction using Vera Rubin NVL72 rack-scale systems. CoreWeave also said it plans to integrate Rubin and operate it through CoreWeave Mission Control.
NVIDIA additionally expanded its collaboration with Red Hat to deliver an AI stack optimized for Rubin across Red Hat Enterprise Linux, OpenShift, and Red Hat AI, signaling a focus on enterprise deployment patterns, not just hyperscale labs. On the model side, many teams will still build and iterate within open-source ecosystems like Hugging Face, then map those workloads to the right serving and training infrastructure.
Discover more cutting-edge AI apps and apps on Appse, your go-to directory for the latest AI innovations.