DeepSeek-V4.1-Flash · DeepSWE · v1.1; Resolved metric as printed in V4.1-Flash card · 成绩与配置

DeepSeek-V4.1-Flash 的 DeepSWE · v1.1; Resolved metric as printed in V4.1-Flash card 记录。本次发布有 13 条成绩记录,含 13 条精确值(可含历史记录),附 13 条来源定位。

成绩与评测条件

DeepSeek-V4.1-Flash · DeepSWE 成绩与配置
指标与成绩配置与评测条件出处
Resolved74.2%原文 74.2 · 单位 %
公布 未确认
DeepSeek-V4.1-Flash reasoning_effort=100 (Max display)
设置
reasoning_effort=100 exactly as reported; temperature=1.0, top_p=0.95. All Instruct results use the maximum effort setting.
未知项
Display label Max maps to the original numeric reasoning_effort=100; it does not replace the raw setting.
工具
允许工具
推理预算
100
框架
mini-SWE
协议
DeepSWE v1.1; mini-SWE harness; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
同一发布检查点;普通单代理;主表和 Table 4 mini-SWE 列的配置与分数一致,尚无逐运行复用依据。报告正文明确 74.2% pass rate;DeepSWE 使用 mini-SWE,而非 DeepSeek Harness Minimal。
在排名中查看此成绩
Resolved65.6%原文 65.6 · 单位 %
公布 未确认
Codex v0.147.0
设置
Codex v0.147.0 standard app-server mode with adapted tool schemas. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Codex v0.147.0; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Codex v0.147.0 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved65.5%原文 65.5 · 单位 %
公布 未确认
OpenCode v1.18.15
设置
OpenCode v1.18.15 build agent with shell and file tools and native task delegation. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; OpenCode v1.18.15; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
OpenCode v1.18.15 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved66.2%原文 66.2 · 单位 %
公布 未确认
Pi v0.84.2
设置
Pi v0.84.2 RPC mode with file and shell tools plus search extension. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Pi v0.84.2; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Pi v0.84.2 scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved74.2%原文 74.2 · 单位 %
公布 未确认
DeepSeek-V4.1-Flash reasoning_effort=100 (Max display)
设置
reasoning_effort=100 exactly as reported; temperature=1.0, top_p=0.95. All Instruct results use the maximum effort setting.
未知项
Display label Max maps to the original numeric reasoning_effort=100; it does not replace the raw setting.
工具
允许工具
推理预算
100
框架
mini-SWE
协议
DeepSWE v1.1; mini-SWE harness; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
同一发布检查点;普通单代理;主表和 Table 4 mini-SWE 列的配置与分数一致,尚无逐运行复用依据。报告正文明确 74.2% pass rate;DeepSWE 使用 mini-SWE,而非 DeepSeek Harness Minimal。
Resolved70.5%原文 70.5 · 单位 %
公布 未确认
DeepSeek Harness Standard
设置
DeepSeek Harness Standard with the full SDK profile and 26 initial tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
DeepSeek Harness Standard
协议
DeepSWE v1.1; DeepSeek Harness Standard; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
DeepSeek Harness Standard scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved67.6%原文 67.6 · 单位 %
公布 未确认
DeepSeek Harness PTC
设置
DeepSeek Harness PTC with run_code and 24 underlying tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
DeepSeek Harness PTC
协议
DeepSWE v1.1; DeepSeek Harness PTC; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
DeepSeek Harness PTC scaffold; Max reasoning effort. 报告正文明确该指标为百分比。 Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved68.4%原文 68.4 · 单位 %
公布 未确认
Claude Code v2.1.105
设置
Claude Code v2.1.105 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Claude Code v2.1.105; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Claude Code v2.1.105; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved68.7%原文 68.7 · 单位 %
公布 未确认
Claude Code v2.1.238
设置
Claude Code v2.1.238 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Claude Code v2.1.238; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Claude Code v2.1.238; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved69.8%原文 69.8 · 单位 %
公布 未确认
Claude Code v2.1.251
设置
Claude Code v2.1.251 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Claude Code v2.1.251; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Claude Code v2.1.251; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved68.6%原文 68.6 · 单位 %
公布 未确认
Claude Code v2.1.259
设置
Claude Code v2.1.259 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
DeepSWE v1.1; Claude Code v2.1.259; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Claude Code v2.1.259; 报告正文明确该指标为百分比; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Resolved68.9%原文 68.9 · 单位 %
公布 未确认
Claude Code four-version average
设置
Average computed from the unrounded Pass@1 scores of four Claude Code versions. Max reasoning effort; same scaffold and benchmark settings as Table 4.
未知项
Claude Code version-specific results are retained raw and are not mapped as ordinary effort tiers.
工具
工具条件未记录
推理预算
Max
框架
Claude Code harness
协议
DeepSWE v1.1; Claude Code four-version average; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Claude Code four-version average; 报告正文明确该指标为百分比.
Resolved72.6%原文 72.6 · 单位 %
公布 未确认
DeepSeek Harness Minimal
设置
DeepSeek Harness Minimal with a single bash tool (Appendix B.1). Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
DeepSeek Harness Minimal
协议
DeepSWE v1.1; DeepSeek Harness Minimal; N=8; Max reasoning_effort=100; temperature=1.0, top_p=0.95; 1M context; max_steps=500.
条件
Table 4 DeepSeek Harness Minimal 列;普通单代理;与 mini-SWE 的74.2分开保留。
在排名中查看此成绩

出处与披露

测试版本与指标

  • Resolved · % · 越高越好