HealthBench Hard

Open-ended health conversations graded against physician-written rubrics.

How each maker reported it

GPT-6 Astra Launch post (openai.com/index/gpt-6-astra); system card §6, §10.1; GPT-6.1 Sol addendum §5, §9
Coding and reasoning rows come from the GPT-6 Astra launch post (3 Sep 2026); OpenAI's own comparison table, settings as published there. The system card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted. GPT-6.1 Sol and some GPT-5.6 Sol values come from the GPT-6.1 Sol addendum (29 Sep 2026).
GPT-6.1 Sol §5, §9
The addendum reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; bio scores are refusal-adjusted. HealthBench scores are length-adjusted.
GPT-6 Luna Appendix A, p.120–154; GPT-6.1 Sol addendum Table 9
The card reports safety and Preparedness evaluations, not general capability benchmarks. Cyber and bio scores are at maximum reasoning effort; HealthBench scores are length-adjusted. MentalHealthBench comes from the GPT-6.1 Sol addendum.
gpt-oss-120b Figures 1, 2, 4; Table 2
OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o4-mini and o3 values read from OpenAI's charts.
gpt-oss-20b Figures 1, 2, 4; Table 2
OpenAI's own pass@1 results at reasoning level high. AIME and HLE with tools; Codeforces with a terminal tool. o3-mini was run without tools on AIME, HLE and Codeforces, so those cells are left empty. Baseline values read from OpenAI's charts.

Only each model's own maker's number is shown; scores a maker reports for rival models are left out.