aiaimodel.cards
← All benchmarks

Terminal-bench 2.1

Completing multi-step tasks in a real terminal: shell commands, builds, data wrangling and debugging, graded automatically.

  1. 1DeepSeek-V4.1-Flash · DeepSeek
    90.6%
  2. 2Gemini 3.8 Flash · Google
    89.4%
  3. 3DeepSeek-V4-Pro · DeepSeek
    87.9%

How each maker reported it

  • DeepSeek-V4.1-Flash Table 3, p.33

    All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

  • Gemini 3.8 Flash p.5

    Results as of September 2026, as published in Google's model card. Prices in the card list $0.75 / $3.75 with $1.50 / $7.50 as the regular rate. LVBench value for 3.8 Flash is the agentic setting (static: 87.1%).

  • DeepSeek-V4-Pro model card (0813)

    Code-agent tasks use DeepSeek Harness minimal mode at max effort, temperature 1.0, top_p 0.95. DSBench rows are DeepSeek internal test sets. Competitor figures as published by DeepSeek.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.