Benchmarks
171 benchmarks reported across the cards. Each score is the maker's own number for its own model, with the card's page reference. Makers use different settings and harnesses, so read cross-maker gaps as rough.
Reported by more than one model · 62
- Humanity's Last Exam (with tools)10 models · top 67.7% · Claude Opus 5.5
- Humanity's Last Exam (no tools)9 models · top 64.4% · Claude Opus 5.5
- HealthBench Professional8 models · top 69.2% · Claude Sonnet 5.5
- AutomationBench6 models · top 59.9% · Mistral Large 4
- DeepSWE v1.16 models · top 74.2% · DeepSeek-V4.1-Flash
- GPQA Diamond6 models · top 94.3% · Gemini 3.1 Pro
- SWE-bench Pro6 models · top 89.9% · Claude Opus 5.5
- Terminal-Bench 4.06 models · top 66.4% · Claude Opus 5.5
- GDPval-AA v25 models · top 1861 Elo · Claude Opus 5
- HealthBench Hard5 models · top 36.6 score · GPT-6 Astra
- SWE-bench Multilingual5 models · top 93.9% · Claude Opus 5.5
- SWE-bench Multimodal5 models · top 61.4% · Claude Opus 5.5
- SWE-bench Verified5 models · top 80.6% · Gemini 3.1 Pro
- ExploitGym4 models · top 42.4% · GPT-6 Astra
- FrontierCode v1.1 (Main)4 models · top 54.4% · Claude Opus 5.5
- SEC-Bench Pro4 models · top 85.4% · GPT-6 Astra
- AA-Briefcase3 models · top 1720 Elo · Claude Opus 5
- AA-Briefcase v1.13 models · top 1822 Elo · Claude Opus 5.5
- ExploitBench3 models · top 100.0% · GPT-6 Astra
- ExploitBench Internal Port (Jun–Aug 2026)3 models · top 31.5% · GPT-6 Astra
- GDPval-AA v2.13 models · top 1846 Elo · Claude Opus 5.5
- HealthBench3 models · top 57.6 score · gpt-oss-120b
- MentalHealthBench (overall)3 models · top 58.7% · GPT-6 Astra
- MMLU Pro3 models · top 85.2% · Gemma 4 31B
- MMMU Pro3 models · top 80.5% · Gemini 3.1 Pro
- Multimodal Troubleshooting Virology3 models · top 63.1% · GPT-6 Astra
- OSWorld 2.0 (partial)3 models · top 81.8% · Claude Opus 5.5
- Terminal-bench 2.13 models · top 90.6% · DeepSeek-V4.1-Flash
- Terminal-Bench-Science 0.13 models · top 64.6% · GPT-6 Astra
- TroubleshootingBench3 models · top 63.5% · GPT-6 Astra
- AA-LCR2 models · top 80.0% · Muse Glimmer 30B
- Agents' Last Exam2 models · top 31.8% · DeepSeek-V4.1-Flash
- AIME 2025 (with tools)2 models · top 98.7% · gpt-oss-20b
- ARC-AGI-12 models · top 97.5% · Claude Fable 5.1
- ARC-AGI-22 models · top 90.4% · Claude Opus 5
- BrowseComp2 models · top 90.8% · Claude Opus 5
- ChartQA2 models · top 90.0% · Llama 4 Maverick
- CharXiv Reasoning (no tools)2 models · top 86.2% · Gemini 3.8 Flash
- Codeforces2 models · top 3471 Elo · DeepSeek-V4.1-Flash
- Codeforces (with tools)2 models · top 2622 Elo · gpt-oss-120b
- CyberGym2 models · top 88.1% · DeepSeek-V4.1-Flash
- DocVQA (test)2 models · top 94.4 ANLS · Llama 4 Scout
- GPQA Diamond (no tools)2 models · top 80.1% · gpt-oss-120b
- Internal Research Debugging2 models · top 78.0% · GPT-6 Astra
- LiveCodeBench (10/2024-02/2025)2 models · top 43.4% · Llama 4 Maverick
- MathVista2 models · top 73.7% · Llama 4 Maverick
- MGSM2 models · top 92.3% · Llama 4 Maverick
- MMLU2 models · top 90.0% · gpt-oss-120b
- MMMLU (14-language average)2 models · top 81.3% · gpt-oss-120b
- MMMU2 models · top 73.4% · Llama 4 Maverick
- MMMU Pro (avg of Standard and Vision)2 models · top 59.6% · Llama 4 Maverick
- MRCR v2 8-needle 128K (average)2 models · top 84.9% · Gemini 3.1 Pro
- MTOB full book, eng→kgv2 models · top 50.8 chrF · Llama 4 Maverick
- MTOB full book, kgv→eng2 models · top 46.7 chrF · Llama 4 Maverick
- MTOB half book, eng→kgv2 models · top 54 chrF · Llama 4 Maverick
- MTOB half book, kgv→eng2 models · top 46.4 chrF · Llama 4 Maverick
- OSWorld 2.0 (strict)2 models · top 48.7% · Claude Opus 5.5
- OSWorld-Verified2 models · top 74.0% · Gemini 3.5 Flash-Lite
- Stylization2 models · top 1098 Elo · Nano Banana Pro
- SWE-Bench Pro (Public)2 models · top 54.2% · Gemini 3.5 Flash-Lite
- Tacit Knowledge and Troubleshooting2 models · top 88.5% · GPT-6.1 Sol
- τ-Bench Retail2 models · top 67.8% · gpt-oss-120b
Reported by one model · 109
- AIME 20261 model · top 94.7% · Muse Glimmer 30B
- AIME 2026 (no tools)1 model · top 89.2% · Gemma 4 31B
- APEX-Agents1 model · top 33.5% · Gemini 3.1 Pro
- ARC-AGI-2 (ARC Prize Verified)1 model · top 77.1% · Gemini 3.1 Pro
- ARC-AGI-31 model · top 30.2% · Claude Opus 5
- AutomationBench (Public)1 model · top 31.8% · DeepSeek-V4-Pro
- Average WER, non-streaming (Artificial Analysis)1 model · top 2.6% · Gemini 3.5 Transcribe
- Average WER, streaming (Artificial Analysis)1 model · top 4.0% · Gemini 3.5 Transcribe
- BabyVision (with tools)1 model · top 89.6% · DeepSeek-V4.1-Flash
- Beam128K1 model · top 65.1% · Muse Glimmer 30B
- BigBench Extra Hard1 model · top 74.4% · Gemma 4 31B
- Blind human coding eval (Surge AI)1 model · top 3.74 /5 · Mistral Large 4
- CADGenBench (generation)1 model · top 44.4% · Grok 4.7
- Character editing1 model · top 1176 Elo · Nano Banana Pro
- Chart editing1 model · top 1209 Elo · Nano Banana Pro
- Chartography (with tools)1 model · top 78.9% · DeepSeek-V4.1-Flash
- CharXiv Reasoning1 model · top 78.8% · Muse Glimmer 30B
- CharXiv Reasoning (with tools)1 model · top 76.5% · Gemini 3.5 Flash-Lite
- Coding Agent Index1 model · top 49.8% · Mistral Large 4
- CursorBench 4.01 model · top 46.3% · Grok 4.7
- Cybench1 model · top 93.0% · Mistral Large 4
- CyberGym (mean reproduced)1 model · top 80.3% · Grok 4.7
- DeepSearch QA1 model · top 74.6% · Muse Glimmer 30B
- DeepSWE1 model · top 62.7% · DeepSeek-V4-Pro
- Dense 200 (visual grounding)1 model · top 42.0% · Mistral Large 4
- Doodle editing1 model · top 1106 Elo · Nano Banana Pro
- DSBench-FullStack (internal)1 model · top 71.1% · DeepSeek-V4-Pro
- DSBench-Hard (internal)1 model · top 67.2% · DeepSeek-V4-Pro
- EEBench1 model · top 66.0% · Grok 4.7
- Factuality – Edu1 model · top 1169 Elo · Nano Banana Pro
- FLEURS WER, non-streaming (top languages)1 model · top 5.0% · Gemini 3.5 Transcribe
- FLEURS WER, streaming (top languages)1 model · top 5.5% · Gemini 3.5 Transcribe
- FrontierBench v0.11 model · top 43.3% · Claude Opus 5
- FrontierCode 1.1 (Main)1 model · top 53.4% · Claude Opus 5
- FrontierSWE V2 (mean@5)1 model · top 29.0% · Grok 4.7
- Gaia21 model · top 43.3% · Muse Glimmer 30B
- GDM-MRCR v2 8-needle, 128K (average)1 model · top 72.2% · Gemini 3.5 Flash-Lite
- GDM-MRCR v2 8-needle, 1M (pointwise)1 model · top 21.3% · Gemini 3.5 Flash-Lite
- GDPval-AA1 model · top 1317 Elo · Gemini 3.1 Pro
- General editing1 model · top 1026 Elo · Nano Banana 2.1
- General image editing1 model · top 1127 Elo · Nano Banana Pro
- General text-to-image1 model · top 1094 Elo · Nano Banana Pro
- GPQA Diamond (AA)1 model · top 83.5% · Muse Glimmer 30B
- Harvey's Legal Agent Benchmark (all pass)1 model · top 10.0% · Gemini 3.8 Flash
- HealthBench Professional (length-adjusted)1 model · top 64.8% · Claude Haiku 5.5
- HLE (with search)1 model · top 26.5% · Gemma 4 31B
- HLE Text (AA)1 model · top 22.0% · Muse Glimmer 30B
- HLE-Verified1 model · top 54.9% · Gemini 3.8 Flash
- Humanity's Last Exam (search + code)1 model · top 51.4% · Gemini 3.1 Pro
- IFBench1 model · top 77.0% · Muse Glimmer 30B
- Infographic design1 model · top 1048 Elo · Nano Banana 2.1
- Infographic factuality1 model · top 0.521 score · Nano Banana 2.1
- Infographics1 model · top 1268 Elo · Nano Banana Pro
- LABBench21 model · top 86.2% · Gemini 3.8 Flash
- LatchBio Capabilities v1.01 model · top 44.5% · Grok 4.7
- Legal Agent Benchmark (Harvey final score)1 model · top 19.6% · Grok 4.7
- LEXam-hard1 model · top 31.89 score · Mistral Medium 3.5
- LiveCodeBench Pro1 model · top 2887 Elo · Gemini 3.1 Pro
- LiveCodeBench v61 model · top 80.0% · Gemma 4 31B
- LVBench1 model · top 87.8% · Gemini 3.8 Flash
- Mask/ink-based editing1 model · top 1049 Elo · Nano Banana 2.1
- MATH-Vision1 model · top 85.6% · Gemma 4 31B
- MathArena Apex1 model · top 65.6% · DeepSeek-V4.1-Flash
- MCP Atlas1 model · top 69.2% · Gemini 3.1 Pro
- MCP Atlas (Public)1 model · top 75.5% · Muse Glimmer 30B
- MedXPertQA MM1 model · top 61.3% · Gemma 4 31B
- MLE-Bench1 model · top 39.2% · Gemini 3.5 Flash-Lite
- MMMLU1 model · top 88.4% · Gemma 4 31B
- Multi-character consistency1 model · top 1106 Elo · Nano Banana 2.1
- Multi-character editing1 model · top 1213 Elo · Nano Banana Pro
- Multi-input (1–3)1 model · top 1109 Elo · Nano Banana Pro
- Multi-reference editing1 model · top 1066 Elo · Nano Banana 2.1
- Multi-turn1 model · top 1186 Elo · Nano Banana Pro
- NL2Repo1 model · top 61.5 Score · DeepSeek-V4-Pro
- NL2Repo-Bench1 model · top 65.4 Score · DeepSeek-V4.1-Flash
- Object/environment editing1 model · top 1102 Elo · Nano Banana Pro
- OmniDocBench 1.5 (edit distance, lower is better)1 model · top 0.131 score · Gemma 4 31B
- OmniDocBench v1.51 model · top 75.8% · Muse Glimmer 30B
- OSWorld 2.01 model · top 70.6% · Claude Opus 5
- OSWorld 2.11 model · top 80.1% · Claude Sonnet 5.5
- OSWorld 2.1 (offline subset)1 model · top 72.4% · Claude Haiku 5.5
- Overall preference1 model · top 1050 Elo · Nano Banana 2.1
- Product consistency1 model · top 1024 Elo · Nano Banana 2.1
- ProgramBench (Almost@1)1 model · top 20.3% · DeepSeek-V4.1-Flash
- Regression sets / popular use cases1 model · top 1128 Elo · Nano Banana Pro
- Reproduce-and-patch1 model · top 82.0% · Mistral Large 4
- SandboxBench1 model · top 45.5% · GPT-6 Astra
- SciCode1 model · top 43.6% · Muse Glimmer 30B
- ScreenSpot Pro1 model · top 75.4% · Muse Glimmer 30B
- Single-character consistency1 model · top 1028 Elo · Nano Banana 2.1
- SkillsBench (with skills)1 model · top 44.3% · Muse Glimmer 30B
- SRE-Bench (pass@4)1 model · top 99.2% · GPT-6 Astra
- SWE-Atlas-QnA1 model · top 59.4% · Mistral Large 4
- SWE-Marathon v1.11 model · top 46.0% · Grok 4.7
- Tau2 (average over 3)1 model · top 76.9% · Gemma 4 31B
- Terminal-Bench 2.0 (Terminus-2)1 model · top 68.5% · Gemini 3.1 Pro
- Terminal-bench 2.1 (Terminus-2)1 model · top 54.0% · Gemini 3.5 Flash-Lite
- Terminal-Bench 3.01 model · top 30.0% · DeepSeek-V4.1-Flash
- Terminal-Bench 41 model · top 28.3% · Mistral Large 4
- TerminalBench 2.1 (terminus2)1 model · top 51.7% · Muse Glimmer 30B
- Text editing1 model · top 1202 Elo · Nano Banana Pro
- Text rendering1 model · top 1198 Elo · Nano Banana Pro
- Toolathlon-Verified1 model · top 74.1% · DeepSeek-V4-Pro
- Vals Finance Agent v21 model · top 61.4% · Gemini 3.8 Flash
- Visual design1 model · top 1104 Elo · Nano Banana Pro
- WildClawBench1 model · top 47.6% · Muse Glimmer 30B
- ZeroBench-main (with tools, Pass@5)1 model · top 49.0% · DeepSeek-V4.1-Flash
- τ3-Banking1 model · top 23.5% · Muse Glimmer 30B
- τ³-Telecom1 model · top 91.4% · Mistral Medium 3.5