‹ PublicAI Index
The LLM benchmark aggregator.
OLMoE 1B-7B Instruct
Ai2 · 7B
Strongest in Safe-prompt compliance (#78 of 82), weakest in Safety (#336 of 336). Above par in 0 of 7 scopes. Among the models it meets almost everywhere, it finishes behind Claude 3.5 Sonnet and Claude 3 Opus.
§ 1 · Profile
What it is good at
Bars run from 50 — the average of the models each source lists — so right of the line is above par. The middle column is the gap to whoever leads that scope.
Safety33.4−28.1#336/3362/3
Safe-prompt compliance28.2−32.2#78/821/1
Secure code36.8−29.7#232/2741/1
Jailbreak resistance26−38.9#262/2721/1
Toxicity avoidance26−33#265/2721/1
Fairness35.2−34.8#294/2992/2
Harm refusal35.4−26.2#295/2992/2
§ 2 · Head to head
What it beats, and what beats it
The same models turn up scope after scope. Counted once: where both were placed, who finished higher. Bars run right for OLMoE 1B-7B Instruct, left for the other.
§ 3 · Sources
Where the numbers come from
2 publications, 11 figures. Every one links to the page it was read from.
HELM Safety · HarmBench 62.9%HELM Safety · SimpleSafetyTests 72.5%HELM Safety · Anthropic Red Team 88.9%HELM Safety · BBQ 41.9%HELM Safety · XSTest 84.3%
Enkrypt · Jailbreak risk 35.4%Enkrypt · Harmful content risk 68.9%Enkrypt · CBRN risk 15.5%Enkrypt · Toxicity risk 35.9%Enkrypt · Bias risk 85%Enkrypt · Insecure code risk 60.4%
Badge
[](https://publicai.io/model-index/m/olmoe-1b-7b-instruct)