‹ PublicAI Index
The LLM benchmark aggregator.
Qwen3.8 27B
Alibaba · 28B · open weights
Strongest in Data analysis (#16 of 31), weakest in Agents (#192 of 298). Above par in 19 of 32 scopes. Among the models it meets almost everywhere, it finishes behind Claude Fable 5.1 and Claude Fable 5 and ahead of GPT-6 Luna and MiniMax M3.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Knowledge55.6−6.9#34/1972/3
Document parsing56.1−7.2#30/831/1
Academic knowledge53.9−10#62/1231/1
Factuality48.8−12—/0✱✱
Coding52.7−13.5#65/1793/6
Agentic coding52.8−14.1#60/1733/5
Code generation57−8.3—/20✱0/1
Professional50.4−12#89/1942/2
Document design49−19.7#20/311/1
Cybersecurity52.6−11.3#28/471/1
Legal53.3−12.6#61/1731/1
Finance52.4−10#80/1621/1
Medical44.2−22.3#121/1491/1
Human preference59.8−9#92/3541/1
Human preference59.8−9#92/3541/1
Core abilities49−16.3#112/1993/4
Data analysis54.1−11.1#16/311/1
Instruction following58.4−15.4#18/621/1
Language45−21.6#46/621/1
General intelligence43.7−25.7#130/1782/3
Long context60.9−3.7—/0✱✱
Reasoning46.9−18.5#121/1955/5
Science58.9−4.6#32/1221/1
Reasoning48.1−20.9#74/1473/3
Mathematics40−23.8#120/1522/2
Expert reasoning55.7−10.7—/0✱✱
Agents46.5−19.9#192/2983/6
Long-horizon work38.2−35.8#26/311/1
Web research36.4−27.5#32/351/2
Knowledge work61.4−9.3#38/2082/2
Computer use37.8−30.3#61/681/2
Tool use68.5−5.5—/81✱0/1
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for Qwen3.8 27B, left for the other.
§ 3 · Sources
Where the numbers come from
11 publications, 51 figures. Every one links to the page it was read from.
LMArena Text 1433
Artificial Analysis Intelligence Index 20
GDPval-AA 1427
AA-Briefcase 1391
ARC-AGI-2 42.4%
LiveBench 54LiveBench · Agentic 15.9LiveBench · Reasoning 80LiveBench · Coding 69.3LiveBench · Browser Use 24.2LiveBench · Data Analysis 76.6LiveBench · Research 61.3LiveBench · Design 72.4
LiveBench 1.0 75.3LiveBench 1.0 · Reasoning 80LiveBench 1.0 · Coding 75.7LiveBench 1.0 · Agentic Coding 61.4LiveBench 1.0 · Mathematics 86.2LiveBench 1.0 · Data Analysis 76.6LiveBench 1.0 · Language 74.3LiveBench 1.0 · Instruction Following 72.7
SimpleBench 60.2%
Vals · Legal Research Bench 36.06%Vals · LegalBench 82.43%Vals · Finance Agent 48.55%Vals · TaxEval 70.85%Vals · MortgageTax 64.94%Vals · MedCode 28.7%Vals · MedScribe 83.85%Vals · CyberBench 67.8%Vals · SWE-bench Verified 86% (Mini-SWE-agent)Vals · Vibe Code Bench 64.85% (OpenHands)Vals · Code Migration 14.16%Vals · GPQA Diamond 88.89%Vals · MMLU Pro 84.34%Vals · ProofBench 16%
ParseBench · Overall 69.03%ParseBench · Tables 66.82%ParseBench · Charts 68.05%ParseBench · Content faithfulness 88.01%ParseBench · Semantic formatting 59.28%ParseBench · Visual grounding 63.01%
tau3-Banking 48%Terminal-Bench 2.1 79.8%SciCode 44.7%Humanity's Last Exam (without tools) 33.9%GPQA Diamond 90.5%CritPt 5.4%AA-LCR 77.3%AA-Omniscience Accuracy 15.6%AA-Omniscience Non-Hallucination 69.7%
Badge
[](https://publicai.io/model-index/m/qwen3-8-27b)