The LLM benchmark aggregator.

NVIDIA Nemotron 3 Ultra 550B A55B

NVIDIA · 561B · A55B · open weights

Strongest in Instruction following (#15 of 62), weakest in Agents (#276 of 289). Above par in 13 of 37 scopes. Among the models it meets almost everywhere, it finishes behind Grok 4.7 and Claude Fable 5 and ahead of Llama 4 Maverick Instruct and GPT-4.1 Nano.

§ 1 · Profile

What it is good at

Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.

Knowledge52.4−10.4#79/1901/3

Academic knowledge55.7−8.2#54/1231/1

Factuality52.6−6.4—/0✱✱

Human preference59.2−9.4#99/3461/1

Reasoning47.3−17.5#114/1894/5

Science57−6.5#44/1221/1

Mathematics49.3−14.7#87/1491/2

Reasoning41.5−25.2#113/1432/3

Expert reasoning49.6−11.3—/0✱✱

Professional45.4−17.7#144/1872/2

Document design29.9−37.2#44/471/1

Finance51.1−11.4#89/1611/1

Medical46.5−20#107/1491/1

Legal43.2−23.2#124/1621/1

Coding36.2−30.2#175/1763/6

Agentic coding35.7−31.3#167/1693/5

Code generation52.8−12.5—/20✱0/1

Core abilities41.1−24.1#206/2253/4

Instruction following59.6−14.2#15/621/1

Data analysis27.6−38.3#43/471/1

Language40.4−26.2#56/621/1

General intelligence38.9−29.9#191/2142/3

Long context53.2−11.4—/0✱✱

Safety46.9−14.5#255/3311/3

Secure code63.3−3#27/2731/1

Toxicity avoidance52.3−6.7#166/2711/1

Fairness43.9−26.1#220/2961/2

Jailbreak resistance32.7−32.3#245/2711/1

Harm refusal44.8−16.7#249/2961/2

Agents40.3−25.6#276/2893/6

Web research31.7−32#45/481/2

Long-horizon work33.5−40.5#45/471/1

Computer use33.4−34.5#77/801/2

Knowledge work51.9−19.9#82/1982/2

Tool use34.2−39.8—/81✱0/1

Workflow automation35.6−26.4—/0✱✱

§ 2 · Head to head

What it beats, and what beats it

The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for NVIDIA Nemotron 3 Ultra 550B A55B, left for the other.

GPT-4o Miniwon 15 · lost 9

Llama 4 Scout Instructwon 14 · lost 9

DeepSeek V3won 15 · lost 10

Command Awon 13 · lost 10

Claude Haiku 4.5won 16 · lost 14

GPT-4.1 Nanowon 13 · lost 12

Llama 4 Maverick Instructwon 13 · lost 12

DeepSeek R1won 12 · lost 12

Gemma 4 31Bwon 15 · lost 16

GPT-4owon 11 · lost 12

O3 Miniwon 11 · lost 12

Kimi K2won 11 · lost 13

GLM-4.6won 11 · lost 13

GPT OSS 120Bwon 10 · lost 14

Grok 4won 10 · lost 14

GPT-4.1won 10 · lost 14

GPT-4.1 Miniwon 10 · lost 15

DeepSeek V3.2won 10 · lost 15

Claude Sonnet 4won 10 · lost 16

Gemini 2.5 Flashwon 9 · lost 16

Qwen3 235B A22B Instructwon 8 · lost 16

O4 Miniwon 8 · lost 17

Grok 4.3won 7 · lost 17

GPT-5 Miniwon 7 · lost 18

Claude Opus 4won 6 · lost 17

DeepSeek V4 Flashwon 7 · lost 22

Gemini 2.5 Prowon 6 · lost 19

Mistral Large 4won 6 · lost 22

Kimi K2.6won 5 · lost 20

Gemini 3.6 Flashwon 5 · lost 20

O3won 5 · lost 21

Gemini 3.5 Flashwon 5 · lost 21

MiniMax M3won 7 · lost 30

DeepSeek V4 Prowon 5 · lost 24

Qwen3.8 27Bwon 5 · lost 25

GLM-5.2won 5 · lost 27

GPT-6 Solwon 4 · lost 25

GPT-5.6 Lunawon 5 · lost 32

GLM-5.3 Flashwon 4 · lost 27

GPT-5.6 Solwon 4 · lost 29

Muse Spark 1.1won 3 · lost 22

Claude Sonnet 4.5won 3 · lost 23

Claude Sonnet 4.6won 3 · lost 26

Grok 4.5won 3 · lost 28

Inklingwon 3 · lost 34

GPT-5.6 Terrawon 3 · lost 34

Kimi K2.5won 2 · lost 23

Qwen3.7 Maxwon 2 · lost 23

Claude Opus 4.7won 2 · lost 24

DeepSeek V4.1 Flashwon 2 · lost 25

GLM-5.3won 2 · lost 25

Kimi K3won 2 · lost 25

GPT-6 Lunawon 2 · lost 27

Claude Opus 4.8won 2 · lost 29

GPT-5.5won 2 · lost 29

Gemini 3.8 Flashwon 2 · lost 29

Claude Sonnet 5won 2 · lost 35

Claude Sonnet 5.5won 1 · lost 22

GPT-5won 1 · lost 23

Muse Spark 1.2won 1 · lost 23

Claude Fable 5.1won 1 · lost 24

GPT-5.2won 1 · lost 25

GPT-5.4won 1 · lost 25

Mimo V2.6 Prowon 1 · lost 25

Gemini 3.1 Prowon 1 · lost 25

Claude Opus 5.5won 1 · lost 28

Grok 4.6won 1 · lost 30

Qwen3.8 Maxwon 1 · lost 30

Claude Opus 5won 1 · lost 32

GPT-6.1 Solwon 0 · lost 23

GPT-6 Astrawon 0 · lost 24

Gemini 3.7 Flashwon 0 · lost 25

Claude Fable 5won 0 · lost 25

Grok 4.7won 0 · lost 28

§ 3 · Sources

Where the numbers come from

11 publications, 61 figures. Every one links to the page it was read from.

LMArena Text ↗1 measureread 2026-10-19

LMArena Text 1426

Artificial Analysis ↗1 measureread 2026-10-19

Artificial Analysis Intelligence Index 23

GDPval-AA ↗1 measureread 2026-10-19

GDPval-AA 1013

AA-Briefcase ↗1 measureread 2026-10-19

AA-Briefcase 876

LiveBench ↗8 measuresread 2026-10-19

LiveBench 40.8LiveBench · Agentic 11.2LiveBench · Reasoning 74.7LiveBench · Coding 43.6LiveBench · Browser Use 0LiveBench · Data Analysis 54.5LiveBench · Research 43.2LiveBench · Design 58.6

LiveBench 1.0 ↗8 measuresread 2026-10-19

LiveBench 1.0 67.4LiveBench 1.0 · Reasoning 74.7LiveBench 1.0 · Coding 70.7LiveBench 1.0 · Agentic Coding 38.7LiveBench 1.0 · Mathematics 88.7LiveBench 1.0 · Data Analysis 54.5LiveBench 1.0 · Language 70.8LiveBench 1.0 · Instruction Following 73.4

SimpleBench ↗1 measureread 2026-10-19

SimpleBench 41.7%

Vals.ai ↗12 measuresread 2026-10-19

Vals · Legal Research Bench 15.38%Vals · LegalBench 82.07%Vals · Harvey Legal Agent Benchmark 0.42%Vals · Finance Agent 37.67%Vals · CorpFin 65.46%Vals · TaxEval 73.1%Vals · MedCode 38.62%Vals · SWE-bench Verified 69% (Mini-SWE-agent)Vals · Vibe Code Bench 7.64% (OpenHands)Vals · Code Migration 4.91%Vals · GPQA Diamond 86.11%Vals · MMLU Pro 85.76%

Enkrypt AI Safety Leaderboard ↗6 measuresread 2026-10-19

Enkrypt · Jailbreak risk 27.2%Enkrypt · Harmful content risk 9.4%Enkrypt · CBRN risk 52.7%Enkrypt · Toxicity risk 4.6%Enkrypt · Bias risk 87.3%Enkrypt · Insecure code risk 6.2%

✱✱ IFM — K2 Horizon launch ↗the model’s own publisher17 measuresread 2026-10-19

GDPVal-AA 1162tau3-Banking 14.2%Toolathlon Verified 34.3%Automation Bench Public 8%Apex-Agents (pass@1) 9%MCPMark 45.7%BrowseComp 44.4%WildClawBench 34.2%Terminal-Bench 2.1 53.9%SciCode 39.9%SWE Bench Pro 38.7%Humanity's Last Exam (without tools) 28.4%GPQA Diamond 86.7%CritPt 3.1%AA-LCR 71%AA-Omniscience Accuracy 23%AA-Omniscience Non-Hallucination 70%

✱ CellCog — K2 Horizon analysis ↗5 measuresread 2026-10-19

GDPVal-AA (Elo) 1162Toolathlon Verified 34.3%Terminal-Bench 2.1 53.9%SWE-bench Pro (strict) 38.7%MCPMark 45.7%

Badge

[![PublicAI Index](https://publicai.io/model-index/badge?model=nvidia-nemotron-3-ultra-550b-a55b)](https://publicai.io/model-index/m/nvidia-nemotron-3-ultra-550b-a55b)