NVIDIA Vera Rubin Leads MLPerf Training v6.1 and Agentic Inference Benchmarks
NVIDIA's Vera Rubin platform swept MLPerf Training v6.1 and posted over 10x throughput-per-watt gains on MLPerf Endpoints agentic AI benchmarks.

NVIDIA's new Vera Rubin platform has posted clean sweeps across the MLPerf Training v6.1 benchmark suite while making its debut on the MLPerf Endpoints v1.0 evaluation for agentic inference. The latest benchmark results highlight substantial gains in training speed, multi-GPU scaling efficiency, and energy-conscious token generation over previous hardware generations.
Training Sweeps and Hardware Scaling
Across the MLPerf Training v6.1 submission round, NVIDIA was the only platform to submit results for every benchmark workload. The tests spanned large language models, image generation, recommender systems, and reinforcement learning post-training. In its debut, the Vera Rubin NVL72 configuration reached up to double the performance per GPU compared to the optimized NVIDIA GB300 NVL72 platform.
Demonstrating efficiency at scale, Vera Rubin NVL72 achieved approximately 97 percent scaling efficiency on the DeepSeek-V3 benchmark when expanding compute from 256 to 1,024 GPUs. Quadrupling the hardware footprint cut total training time from 16 minutes down to about four minutes. Large-scale benchmarks reached 10,240 GPUs, with NVIDIA running GB300 systems and Microsoft Azure deploying 10,240 GPUs on GB200 systems. Inter-gpu communications were handled by NVIDIA's NVLink scale-up networking within systems, combined with Quantum-X800 InfiniBand and Spectrum-X Ethernet scale-out networking across data center hardware.
Agentic Inference and Energy Efficiency
Beyond raw training speed, MLPerf Endpoints v1.0 measures performance on multi-step agentic workflows, evaluating tasks such as software coding, tool execution, and automated reasoning under real-world conditions. Because power availability limits AI data centers, token generation throughput per watt serves as a key measure of economic and operational output.
In provisional results using the Kimi K3 open agentic model, the Vera Rubin NVL72 architecture achieved more than 10 times the throughput per watt of the GB300 NVL72 while maintaining identical response interactivity. A 32-accelerator Vera Rubin NVL72 setup processed 1,024 concurrent queries, generating 41,414.2 tokens per second at 100.89 kW operating power. In comparison, a 48-accelerator GB300 NVL72 running 48 concurrent queries generated 2,210.4 tokens per second at 60 kW power, evaluated at a minimum interactivity threshold of 50 tokens per second per user.
NVIDIA's submission ecosystem included results from 20 partner organizations, such as Google Cloud, Oracle, CoreWeave, Dell Technologies, Supermicro, Cisco, HPE, Lambda, Nebius, and the University of Florida. NVIDIA also reported top performance on third-party benchmarks including SemiAnalysis AgentX and Artificial Analysis AgentPerf.
What it means for developers
For engineering teams and software architects deploying generative models and autonomous agents, these benchmark outcomes outline important practical developments:
Accelerated Model Training: Shorter training times across LLMs, image generators, and post-training reinforcement learning allow teams to retrain models more frequently. This enables faster integration of fresh domain data for applications like fraud detection, scientific research, and enterprise automation.
Scalable Agentic Execution: Modern AI agents frequently make tool calls, generate code, and verify intermediate steps, generating high token volumes per task. Achieving 10x higher throughput per watt allows teams to host larger numbers of active concurrent agents within existing data center power caps.
Flexibility Across Infrastructure Providers: Hardware compatibility across cloud partners—such as Azure, Google Cloud, and specialized providers like CoreWeave and Lambda—means developers can target standardized CUDA environments without lock-in to single physical hardware deployments.
To build and experiment with state-of-the-art foundation models without maintaining complex infrastructure, developers can try top AI models cheaply through one API at https://apixoai.online.
As ongoing CUDA software optimizations continue to enhance hardware output, developers can expect continuous improvements in latency and concurrent capacity across training and inference workloads.
Source: NVIDIA Sweeps MLPerf Training and Endpoints as Vera Rubin Debuts With Leading Performance — NVIDIA blog. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key

