| Resolved74.2%Reported 74.2 · Unit % Published Unconfirmed | DeepSeek-V4.1-Flash reasoning_effort=100 (Max display)- Settings
- reasoning_effort=100 exactly as reported; temperature=1.0, top_p=0.95. All Instruct results use the maximum effort setting.
- Unknowns
- Display label Max maps to the original numeric reasoning_effort=100; it does not replace the raw setting.
- Tools
- Tools allowed
- Reasoning budget
- 100
- Framework
- mini-SWE
- Protocol
- DeepSWE v1.1; mini-SWE harness; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- 同一发布检查点;普通单代理;主表和 Table 4 mini-SWE 列的配置与分数一致,尚无逐运行复用依据。报告正文明确 74.2% pass rate;DeepSWE 使用 mini-SWE,而非 DeepSeek Harness Minimal。
| View this score in rankings |
|---|
| Resolved65.6%Reported 65.6 · Unit % Published Unconfirmed | Codex v0.147.0- Settings
- Codex v0.147.0 standard app-server mode with adapted tool schemas. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Codex v0.147.0; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Codex v0.147.0 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved65.5%Reported 65.5 · Unit % Published Unconfirmed | OpenCode v1.18.15- Settings
- OpenCode v1.18.15 build agent with shell and file tools and native task delegation. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; OpenCode v1.18.15; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- OpenCode v1.18.15 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved66.2%Reported 66.2 · Unit % Published Unconfirmed | Pi v0.84.2- Settings
- Pi v0.84.2 RPC mode with file and shell tools plus search extension. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Pi v0.84.2; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Pi v0.84.2 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved74.2%Reported 74.2 · Unit % Published Unconfirmed | DeepSeek-V4.1-Flash reasoning_effort=100 (Max display)- Settings
- reasoning_effort=100 exactly as reported; temperature=1.0, top_p=0.95. All Instruct results use the maximum effort setting.
- Unknowns
- Display label Max maps to the original numeric reasoning_effort=100; it does not replace the raw setting.
- Tools
- Tools allowed
- Reasoning budget
- 100
- Framework
- mini-SWE
- Protocol
- DeepSWE v1.1; mini-SWE harness; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- 同一发布检查点;普通单代理;主表和 Table 4 mini-SWE 列的配置与分数一致,尚无逐运行复用依据。报告正文明确 74.2% pass rate;DeepSWE 使用 mini-SWE,而非 DeepSeek Harness Minimal。
| |
|---|
| Resolved70.5%Reported 70.5 · Unit % Published Unconfirmed | DeepSeek Harness Standard- Settings
- DeepSeek Harness Standard with the full SDK profile and 26 initial tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- DeepSeek Harness Standard
- Protocol
- DeepSWE v1.1; DeepSeek Harness Standard; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- DeepSeek Harness Standard scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved67.6%Reported 67.6 · Unit % Published Unconfirmed | DeepSeek Harness PTC- Settings
- DeepSeek Harness PTC with run_code and 24 underlying tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- DeepSeek Harness PTC
- Protocol
- DeepSWE v1.1; DeepSeek Harness PTC; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- DeepSeek Harness PTC scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved68.4%Reported 68.4 · Unit % Published Unconfirmed | Claude Code v2.1.105- Settings
- Claude Code v2.1.105 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Claude Code v2.1.105; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Claude Code v2.1.105; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved68.7%Reported 68.7 · Unit % Published Unconfirmed | Claude Code v2.1.238- Settings
- Claude Code v2.1.238 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Claude Code v2.1.238; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Claude Code v2.1.238; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved69.8%Reported 69.8 · Unit % Published Unconfirmed | Claude Code v2.1.251- Settings
- Claude Code v2.1.251 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Claude Code v2.1.251; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Claude Code v2.1.251; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved68.6%Reported 68.6 · Unit % Published Unconfirmed | Claude Code v2.1.259- Settings
- Claude Code v2.1.259 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- not recorded
- Protocol
- DeepSWE v1.1; Claude Code v2.1.259; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Claude Code v2.1.259; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
| View this score in rankings |
|---|
| Resolved68.9%Reported 68.9 · Unit % Published Unconfirmed | Claude Code four-version average- Settings
- Average computed from the unrounded Pass@1 scores of four Claude Code versions. Max reasoning effort; same scaffold and benchmark settings as Table 4.
- Unknowns
- Claude Code version-specific results are retained raw and are not mapped as ordinary effort tiers.
- Tools
- Tool access not recorded
- Reasoning budget
- Max
- Framework
- Claude Code harness
- Protocol
- DeepSWE v1.1; Claude Code four-version average; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Claude Code four-version average; 报告正文明确该指标为百分比.
| |
|---|
| Resolved72.6%Reported 72.6 · Unit % Published Unconfirmed | DeepSeek Harness Minimal- Settings
- DeepSeek Harness Minimal with a single bash tool (Appendix B.1). Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
- Unknowns
- Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
- Tools
- Tools allowed
- Reasoning budget
- Max
- Framework
- DeepSeek Harness Minimal
- Protocol
- DeepSWE v1.1; DeepSeek Harness Minimal; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
- Conditions
- Table 4 DeepSeek Harness Minimal 列;普通单代理;与 mini-SWE 的74.2分开保留。
| View this score in rankings |
|---|