GPQA Diamond

Graduate-level science multiple-choice questions written to be hard to look up.

How each maker reported it

Gemini 3.1 Pro p.4
Results as of February 2026, Gemini at Thinking (High); competitors at their highest reported settings, as listed by Google.
DeepSeek-V4.1-Flash Table 3, p.33
All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.
Gemma 4 31B Benchmark results
Instruction-tuned models, as published in Google's model card. OmniDocBench is edit distance (lower is better). Audio benchmarks are omitted because 31B has no audio input.
Mistral Small 4 HF model card, launch post
Vendor-reported. With reasoning, Mistral says it matches or beats GPT-OSS 120B on AA LCR, LiveCodeBench and AIME25 with shorter outputs; those comparisons are charts without numbers.
Llama 4 Maverick Benchmarks: Instruction tuned models
Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.
Llama 4 Scout Benchmarks: Instruction tuned models
Meta's own 0-shot results on bf16 weights. Llama 3.x models have no image support and a 128K context, so those cells are not reported.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.