SWE-bench Verified with a prompt modification; the main article reports 81.42%.
未知项
The footnote does not explicitly confirm that the 25-trial average or the main result's effort/sampling conditions also apply to this prompt variant.
工具
工具条件未记录
推理预算
未记录
框架
未记录
协议
SWE-bench Verified; prompt-modified variant.
条件
The article reports 81.42% after a prompt modification. Its adjacent statement says the primary score averaged 25 trials, but does not explicitly tie that trial count or the primary result's effort/sampling settings to this variant.
System Card §8.1 standard configuration: adaptive thinking at max effort; default temperature/top_p; averaged over 5 trials. Context sizes are evaluation-dependent and do not exceed 1M tokens; retain all section-specific exceptions on their results.
未知项
Per-benchmark tools, context/token cap, resolution, trial count, harness and other exceptions are recorded on each result when stated.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
SWE-bench Verified。
条件
500-problem Verified subset; §8.1 standard adaptive-thinking/max-effort configuration; average over five trials. The announcement also describes memorization screening and exclusions.
Adaptive thinking · max effort (card-wide default)
设置
System Card §8.1 default: adaptive thinking at max effort, default sampling settings (temperature/top_p), averaged over five trials unless otherwise noted; context varies by evaluation and does not exceed 1M tokens.
未知项
Per-benchmark exceptions and harness details are recorded on the relevant configurations/results.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
SWE-bench Verified; 500-problem subset.
条件
Five-trial average; card-wide default adaptive max effort and default sampling.
Claude Opus · SWE-bench Verified · Verified subset (500 problems) · Release history
View scores, configurations, and disclosures for SWE-bench Verified · Verified subset (500 problems) across 5 Claude Opus releases, ordered by release date.
Differences are calculated only for unique, active, exact values between direct predecessors when the unit, metric, and explicitly recorded configuration and conditions match.
SWE-bench Verified with a prompt modification; the main article reports 81.42%.
Unknowns
The footnote does not explicitly confirm that the 25-trial average or the main result's effort/sampling conditions also apply to this prompt variant.
Tools
Tool access not recorded
Reasoning budget
not recorded
Framework
not recorded
Protocol
SWE-bench Verified; prompt-modified variant.
Conditions
The article reports 81.42% after a prompt modification. Its adjacent statement says the primary score averaged 25 trials, but does not explicitly tie that trial count or the primary result's effort/sampling settings to this variant.
· 1 score record for this release, including 1 exact value (may include historical records)
Records for the direct predecessor do not provide matching conditions for a valid difference.
Claude Opus 4.7 · SWE-bench Verified scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Pass rate87.6%Reported 87.6% · Unit %
Published Unconfirmed
发布表报告值
Settings
System Card §8.1 standard configuration: adaptive thinking at max effort; default temperature/top_p; averaged over 5 trials. Context sizes are evaluation-dependent and do not exceed 1M tokens; retain all section-specific exceptions on their results.
Unknowns
Per-benchmark tools, context/token cap, resolution, trial count, harness and other exceptions are recorded on each result when stated.
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
SWE-bench Verified。
Conditions
500-problem Verified subset; §8.1 standard adaptive-thinking/max-effort configuration; average over five trials. The announcement also describes memorization screening and exclusions.
· 1 score record for this release, including 1 exact value (may include historical records)
Records for the direct predecessor do not provide matching conditions for a valid difference.
Claude Opus 5 · SWE-bench Verified scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Pass rate96%Reported 96.0% · Unit %
Published Unconfirmed
Adaptive thinking · max effort (card-wide default)
Settings
System Card §8.1 default: adaptive thinking at max effort, default sampling settings (temperature/top_p), averaged over five trials unless otherwise noted; context varies by evaluation and does not exceed 1M tokens.
Unknowns
Per-benchmark exceptions and harness details are recorded on the relevant configurations/results.
Tools
Tool access not recorded
Reasoning budget
max
Framework
not recorded
Protocol
SWE-bench Verified; 500-problem subset.
Conditions
Five-trial average; card-wide default adaptive max effort and default sampling.