aiaimodel.cards
← All benchmarks

FrontierCode v1.1 (Main)

Agentic coding tasks taken from real pull requests, graded on hidden tests plus code-quality criteria.

  1. 1Claude Opus 5.5 · Anthropic
    54.4%
  2. 2GPT-6 Astra · OpenAI
    53.3%
  3. 3Claude Haiku 5.5 · Anthropic
    46.4%
  4. 4Claude Sonnet 5.5 · Anthropic
    46.2%

How each maker reported it

  • Claude Opus 5.5 Table 8.1.A, p.174

    Claude results use adaptive thinking at max effort, averaged over five trials (Terminal-Bench 4.0 at xhigh). Competitor figures come from the developers' own published results.

  • GPT-6 Astra Launch post (openai.com/index/gpt-6-astra); system card §6, §10.1; GPT-6.1 Sol addendum §5, §9

    Coding and reasoning rows come from the GPT-6 Astra launch post (3 Sep 2026); OpenAI's own comparison table, settings as published there. The system card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted. GPT-6.1 Sol and some GPT-5.6 Sol values come from the GPT-6.1 Sol addendum (29 Sep 2026).

  • Claude Haiku 5.5 Table 8.1.A, p.111

    Haiku 5.5 results use adaptive thinking at max effort with default sampling, averaged over five trials. FrontierCode is shown at max effort (Sonnet 5.5 scores 52.1 at xhigh). OSWorld is the offline subset; GPT-6 Luna was run by Anthropic on the same tasks. Other competitor figures come from the developers' own published results.

  • Claude Sonnet 5.5 Table 8.1.A, p.109

    Claude results use adaptive thinking at max effort with default sampling, averaged over five trials. Competitor figures come from the developers' own published system cards or leaderboards.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.