‹ PublicAI Index
The LLM benchmark aggregator.
Claude 3.5 Haiku
Anthropic
Strongest in Jailbreak resistance (#5 of 268), weakest in Agents (#221 of 298). Above par in 5 of 20 scopes. Among the models it meets almost everywhere, it finishes behind Claude Fable 5 and Claude Fable 5.1 and ahead of Gemma 4 31B and Llama 3.3 70B Instruct.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Safety58.7−2.7#12/3291/3
Jailbreak resistance64−1#5/2681/1
Harm refusal60−1.5#8/2921/2
Fairness70leads#9/2921/2
Toxicity avoidance58.1−0.9#25/2681/1
Secure code40.6−25.6#215/2701/1
Coding46.6−19.6#109/1791/6
Agentic coding46.3−20.6#105/1731/5
Professional43.6−18.8#164/1941/2
Legal43.9−22#128/1731/1
Finance39.6−22.8#141/1621/1
Knowledge41.1−21.4#175/1971/3
Academic knowledge28.6−35.3#117/1231/1
Reasoning33.8−31.6#192/1951/5
Science26.1−37.4#116/1221/1
Mathematics33.3−30.5#147/1521/2
Human preference49.5−19.3#204/3541/1
Human preference49.5−19.3#204/3541/1
Agents44.9−21.5#221/2981/6
Knowledge work39.9−30.8#154/2081/2
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for Claude 3.5 Haiku, left for the other.
§ 3 · Sources
Where the numbers come from
5 publications, 15 figures. Every one links to the page it was read from.
LMArena Text 1325
GDPval-AA 218
Aider polyglot 28%
Vals · LegalBench 70.33%Vals · CorpFin 50.82%Vals · TaxEval 57.36%Vals · GPQA Diamond 37.88%Vals · MMLU Pro 64.12%Vals · AIME 3.33%
Enkrypt · Jailbreak risk 1.1%Enkrypt · Harmful content risk 5.6%Enkrypt · CBRN risk 3%Enkrypt · Toxicity risk 0.6%Enkrypt · Bias risk 43.4%Enkrypt · Insecure code risk 52.4%
Badge
[](https://publicai.io/model-index/m/claude-3-5-haiku)