‹ PublicAI Index
The LLM benchmark aggregator.
GPT-4o mini
OpenAI
Strongest in Multimodal understanding (#6 of 19), weakest in Agents (#250 of 298). Above par in 7 of 26 scopes. Among the models it meets almost everywhere, it finishes behind Claude Sonnet 4 and GPT-4o and ahead of Llama 3.3 70B Instruct and Llama 4 Scout Instruct.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Safety53−8.4#102/3292/3
Safe-prompt compliance54.2−6.2#39/821/1
Jailbreak resistance58.2−6.8#64/2681/1
Toxicity avoidance56.1−2.9#70/2681/1
Secure code57.1−9.1#94/2701/1
Harm refusal53.1−8.4#106/2922/2
Fairness46.4−23.6#157/2922/2
Knowledge45.1−17.4#151/1972/3
Multimodal understanding57.3−14.8#6/191/1
Academic knowledge26.9−37#119/1231/1
Professional43.1−19.3#167/1941/2
Medical44.1−22.4#122/1491/1
Finance39.3−23.1#142/1621/1
Core abilities41.8−23.5#174/1991/4
General intelligence38.2−31.2#162/1781/3
Coding37.9−28.3#177/1792/6
Code generation32.2−33.1#18/201/1
Agentic coding40.6−26.3#157/1731/5
Reasoning32.4−33#195/1953/5
Science27.8−35.7#114/1221/1
Mathematics35.4−28.4#140/1521/2
Reasoning32.5−36.5#145/1472/3
Human preference48.8−20#214/3541/1
Human preference48.8−20#214/3541/1
Agents43−23.4#250/2981/6
Knowledge work36−34.7#176/2081/2
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for GPT-4o mini, left for the other.
§ 3 · Sources
Where the numbers come from
11 publications, 26 figures. Every one links to the page it was read from.
LMArena Text 1318
GDPval-AA -15
ARC-AGI-2 0%
LiveCodeBench 27.5%
Aider polyglot 3.6%
MMMU-Pro 37.6%
Kagi LLM Benchmark 28.8%
SimpleBench 10.7%
Vals · CorpFin 45.45%Vals · TaxEval 60.55%Vals · MortgageTax 54.49%Vals · MedQA 72.44%Vals · GPQA Diamond 44.19%Vals · MMLU Pro 62.73%Vals · AIME 11.46%
HELM Safety · HarmBench 84.9%HELM Safety · SimpleSafetyTests 97.8%HELM Safety · Anthropic Red Team 98.3%HELM Safety · BBQ 88.2%HELM Safety · XSTest 96%
Enkrypt · Jailbreak risk 5.9%Enkrypt · Harmful content risk 39.4%Enkrypt · CBRN risk 8%Enkrypt · Toxicity risk 2%Enkrypt · Bias risk 86.3%Enkrypt · Insecure code risk 18.7%
Badge
[](https://publicai.io/model-index/m/gpt-4o-mini)