Home Model series Claude Sonnet Claude Sonnet · ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Release history Claude Sonnet · ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 · Release history View scores, configurations, and disclosures for ProgramBench · 166 个保留的 golden tasks;架构与时限按协议区分 across 2 Claude Sonnet releases, ordered by release date.
Differences are calculated only for unique, active, exact values between direct predecessors when the unit, metric, and explicitly recorded configuration and conditions match.
2026-06-30 · 1 score record for this release, with no exact value
Claude Sonnet 5 · ProgramBench scores and configurations Metric and score Configuration and evaluation conditions Sources Hidden test pass rate 76–86% Reported 76–86% · Unit % Published Unconfirmed
The source does not provide a verifiable exact value.
Standard configuration: adaptive thinking, max effort Settings Adaptive thinking at max effort; default temperature and top_p; five-trial averages unless otherwise stated. Unknowns Context-window size is evaluation-dependent; standard configuration is 1M tokens. Tools Tool access not recorded Reasoning budget max Framework mini-SWE-agent harness Protocol 166 retained tasks; five episodes, each continuing the prior codebase with a fresh context budget up to 1M tokens. Conditions Range reported across tasks/episodes, not a single exact scalar; mini-swe-agent harness without upstream six-hour limit. View this score in rankings
§8.8, PDF p.121 Attachment · Claude Sonnet 5 System Card 2026-09-28 · 1 score record for this release, including 1 exact value (may include historical records)
Records for the direct predecessor do not provide matching conditions for a valid difference.
Claude Sonnet 5.5 · ProgramBench scores and configurations Metric and score Configuration and evaluation conditions Sources Hidden test pass rate 79.7% Reported 79.7% · Unit % Published Unconfirmed
Adaptive thinking, max effort Settings Adaptive thinking enabled; max effort. Unknowns Sampling settings may vary by benchmark; see each result.conditions. Tools Tool access not recorded Reasoning budget max Framework not recorded Protocol ProgramBench, 166 tasks after excluding 34 flaky reference binaries; hidden-test pass rate over five episodes. Conditions mini-swe-agent; no six-hour upstream time limit; fresh context budget up to 1M tokens per episode. View this score in rankings
§8.10.1, PDF p.118 Attachment · Claude Sonnet 5.5 System Card §8.12.1, PDF p.123, Figures 8.12.1.A-B Attachment · Claude Sonnet 5.5 System Card