Nvidia 1,000 Tokens a Second Claim, Checked

The headline says Nvidia topped 1,000 tokens a second per user on models above 700B parameters. The fastest per-user figure in the primary record is 826, measured on four B300 GPUs with a decode path built for a single user. No document names a Rubin NVL72 run, an agent swarm, or any result past 1,000.

Verdict: unproven. The five primary documents carry no Nvidia submission, and the fastest per-user figure in them is 826 tokens/s.

The closest source stops at 826

vLLM's blog post, dated October 23, 2026, reports the 826 as a P90 on real AgentX sessions. The standard path in the same post reaches 575, so the specialized path sits 44% above it. On one GB300 NVL72 rack, the vLLM post reports 954K tokens/s per GPU at a P90 of 196 tokens/s per user.

Neither figure reaches 1,000. Another document in the set reports interactivity of up to 376 tokens per second on MiniMax M3. The InferenceX page from SemiAnalysis stops per-user targets at 200 tok/s, counted from the ladder of 30, 50, 75, 100, 150 and 200 on that page.

The vLLM post comes from Inferact. It states "This work was led by Inferact," and thanks six NVIDIA engineers for "close collaboration on performance optimization." My reading is the coverage folded a joint effort into one company's name.

Details the coverage carries and no document contains

The coverage quotes 20.9 times performance per TCO, 325 tokens per second per user, 943 tokens per second at concurrency of 1, a kernel called MegaKernel and a collaboration with RadixArk. None appears in the five documents. The vLLM post names MegaAttn, MegaMoE and Mega-mHC instead.

Two more figures lack a source: 654,000 tokens per second per GPU "four days earlier," and 1.09 million credited to SGLang. InferenceX lists a peak of 1,094,347 with no engine and no date attached. SGLang's own document says DeepSeek-V4.1 Flash support "has not shipped in an SGLang release yet" and reports no throughput figures.

The coverage also reports $38 in revenue per GPU hour. The vLLM post gives $38.96 per GPU-hour "at DeepSeek's off-peak prices" and calls it neither revenue, profit nor cost.

One headline, three different measurements

Start with size. DeepSeek-V4.1-Flash has "552B backbone parameters," short of 700B+. The only model in the documents above 700B is Kimi K3, at 2.8 trillion parameters, and it ran at 62.7 tokens/s/user at P90 on 16 GB300s.

Now the metric. The 954K figure counts "cached input, uncached input and output tokens." Median input in AgentX is 142K tokens against a median output of 444, with a theoretical cache hit rate of more than 96%. I infer most tokens in a per-GPU total are input served from cache, so the figure says little about how fast one person's answer streams.

Per-user speed, per-GPU total throughput and parameter count are three different quantities. The coverage mixes all three.

Five documents without Rubin do not prove the run never happened

The strongest objection is plain. All five documents date from October 2026, and one is a page re-benchmarked continuously. A Rubin NVL72 submission might sit outside them, and a record silent on it shows nothing about whether it exists. I concede this. The defensible claim is that the headline is unverified.

A second objection hits my own comparison. Setting Kimi K3's 62.7 against 1,000 pits one operating point against another. The vLLM post shows how far per-user speed moves: 196 at P90 on a rack reported for per-GPU throughput, 826 on a decode path built for one user. So the 62.7 figure is context. It tells you nothing about what a latency-tuned run on a 700B+ model would reach.

Two points, 575 and 826, also leave open whether a newer rack reached 1,000. InferenceX adds a caution of its own. Engines are rebuilt and re-benchmarked continuously, so every figure on it is a snapshot.

Settling it takes an Nvidia document stating the model, the concurrency, the metric definition and the TCO basis. The record holds none of the four.

The two documents with numbers disagree

On the same model and rack, the vLLM post reports 954K tokens/s per GPU at a P90 of 196 tokens/s per user. InferenceX lists 499,715 at 150 tok/s for Dynamo SGLang and 471,578 at 200 tok/s for Dynamo vLLM, plus a peak of 1,094,347 with no interactivity stated. Neither document reconciles the numbers.

My reading is three causes are possible: differing metric definitions, run dates (October 23 against runs from 2026-10-02 to 2026-10-30), or the ladder's selection method. No document says which.

I started by looking for the number 1,000 and ended up reading conditions. The vLLM post attaches a rack, a model, a percentile and a price basis to its headline figure, so a reader checks each one. The Nvidia headline arrives with none attached.

The one peak SemiAnalysis lists, 1,094,347 tokens/s per GPU, names no engine and states no per-user speed, so it settles nothing in either direction.