‹ PublicAI Index
The LLM benchmark aggregator.
O4 Mini
OpenAI
Strongest in Code generation (#1 of 20), weakest in Toxicity avoidance (#203 of 268). Above par in 19 of 27 scopes. Among the models it meets almost everywhere, it finishes behind Claude Opus 5.5 and Claude Fable 5 and ahead of DeepSeek V3.2 and DeepSeek R1.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Coding59.4−6.8#14/1792/6
Code generation65.3leads#1/201/1
Agentic coding56.6−10.3#26/1731/5
Core abilities55.1−10.2#43/1991/4
General intelligence57.3−12.1#39/1781/3
Agents54.6−11.8#62/2981/6
Tool use61.2−12.8#17/811/1
Safety53.5−7.9#85/3293/3
Safe-prompt compliance57.3−3.1#18/821/1
Harm refusal56.8−4.7#36/2922/2
Jailbreak resistance58.8−6.2#60/2681/1
Secure code60.2−6#64/2701/1
Fairness51.9−18.1#94/2922/2
Factual grounding28−42.7#96/1041/1
Toxicity avoidance49.2−9.8#203/2681/1
Professional50.2−12.2#96/1941/2
Finance53.7−8.7#69/1621/1
Legal50.7−15.2#87/1731/1
Medical46.1−20.4#112/1491/1
Reasoning47.8−17.6#116/1953/5
Mathematics54.6−9.2#59/1521/2
Science48.9−14.6#81/1221/1
Reasoning41.9−27.1#112/1472/3
Knowledge49.7−12.8#116/1971/3
Academic knowledge49.2−14.7#81/1231/1
Human preference55.8−13#142/3541/1
Human preference55.8−13#142/3541/1
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for O4 Mini, left for the other.
§ 3 · Sources
Where the numbers come from
11 publications, 29 figures. Every one links to the page it was read from.
LMArena Text 1391
ARC-AGI-2 6.1%
LiveCodeBench 80.2%
Aider polyglot 72%
BFCL v4 53.24%
Kagi LLM Benchmark 67.6%
SimpleBench 38.7%
Vals · LegalBench 79.19%Vals · CorpFin 58.97%Vals · TaxEval 74.78%Vals · MortgageTax 64.83%Vals · MedQA 96.02%Vals · MedCode 33.79%Vals · MedScribe 69.14%Vals · GPQA Diamond 74.5%Vals · MMLU Pro 80.56%Vals · AIME 83.67%
HELM Safety · HarmBench 97%HELM Safety · SimpleSafetyTests 100%HELM Safety · Anthropic Red Team 98.2%HELM Safety · BBQ 94%HELM Safety · XSTest 97.4%
Enkrypt · Jailbreak risk 5.4%Enkrypt · Harmful content risk 15%Enkrypt · CBRN risk 12.7%Enkrypt · Toxicity risk 6.7%Enkrypt · Bias risk 79.8%Enkrypt · Insecure code risk 12.4%
Vectara · Factual consistency 81.4%
Badge
[](https://publicai.io/model-index/m/o4-mini)