← All benchmarks
ProgramBench (Almost@1)
We haven't written a description for this benchmark yet; see the card notes below for how each maker ran it.
How each maker reported it
- DeepSeek-V4.1-Flash Table 3, p.33
All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.
Only each model's own maker's number is shown; scores a maker reports for rival models are left out.