‹ PublicAI Index
The LLM benchmark aggregator.
Claude 3.5 Sonnet
Anthropic
Strongest in Harm refusal (#1 of 292), weakest in Reasoning (#170 of 195). Above par in 12 of 22 scopes. Among the models it meets almost everywhere, it finishes behind Claude Sonnet 5 and Gemini 3.1 Pro and ahead of Inkling and NVIDIA Nemotron 3 Ultra 550B A55B.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Safety60.3−1.1#2/3292/3
Harm refusal61.5leads#1/2922/2
Jailbreak resistance65leads#1/2681/1
Toxicity avoidance58.9−0.1#7/2681/1
Fairness65.2−4.8#22/2922/2
Safe-prompt compliance53.3−7.1#43/821/1
Secure code50.1−16.1#154/2701/1
Knowledge55.7−6.8#33/1972/3
Multimodal understanding71.6−0.5#2/191/1
Academic knowledge46.5−17.4#95/1231/1
Professional49.7−12.7#102/1941/2
Medical49.8−16.7#88/1491/1
Finance49.5−12.9#101/1621/1
Coding47.3−18.9#106/1792/6
Code generation37.7−27.6#15/201/1
Agentic coding51.8−15.1#72/1731/5
Human preference54.2−14.6#157/3541/1
Human preference54.2−14.6#157/3541/1
Reasoning39.6−25.8#170/1952/5
Reasoning45.3−23.7#87/1471/3
Science38.3−25.2#101/1221/1
Mathematics35−28.8#141/1521/2
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for Claude 3.5 Sonnet, left for the other.
§ 3 · Sources
Where the numbers come from
8 publications, 23 figures. Every one links to the page it was read from.
LMArena Text 1374
LiveCodeBench 36.4%
Aider polyglot 51.6%
MMMU-Pro 51.5%
SimpleBench 41.4%
Vals · CorpFin 53.61%Vals · TaxEval 70.16%Vals · MortgageTax 64.07%Vals · MedQA 83.19%Vals · GPQA Diamond 59.34%Vals · MMLU Pro 78.4%Vals · AIME 10%
HELM Safety · HarmBench 98.1%HELM Safety · SimpleSafetyTests 100%HELM Safety · Anthropic Red Team 99.8%HELM Safety · BBQ 94.9%HELM Safety · XSTest 95.6%
Enkrypt · Jailbreak risk 0.2%Enkrypt · Harmful content risk 0%Enkrypt · CBRN risk 1.8%Enkrypt · Toxicity risk 0.1%Enkrypt · Bias risk 35.4%Enkrypt · Insecure code risk 32.9%
Badge
[](https://publicai.io/model-index/m/claude-3-5-sonnet)