aiaimodel.cards
← All benchmarks

Agents' Last Exam

We haven't written a description for this benchmark yet; see the card notes below for how each maker ran it.

  1. 1DeepSeek-V4.1-Flash · DeepSeek
    31.8%
  2. 2DeepSeek-V4-Pro · DeepSeek
    25.7%

How each maker reported it

  • DeepSeek-V4.1-Flash Table 3, p.33

    All models at max reasoning effort. Code-agent benchmarks use DeepSeek Harness Minimal mode with 1M context; DeepSWE uses mini-SWE; visual agent tasks use the Claude Code harness with 512K context. HLE for some models is the text-only subset. The Hugging Face card lists NL2Repo-Bench as 64.0; the report and change log say 65.4.

  • DeepSeek-V4-Pro model card (0813)

    Code-agent tasks use DeepSeek Harness minimal mode at max effort, temperature 1.0, top_p 0.95. DSBench rows are DeepSeek internal test sets. Competitor figures as published by DeepSeek.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.