DeepSeek-V4.1-Flash · Terminal-Bench · 2.1 (Pass@1); across agent scaffolds at Max reasoning effort; unit not printed · 成绩与配置

DeepSeek-V4.1-Flash 的 Terminal-Bench · 2.1 (Pass@1); across agent scaffolds at Max reasoning effort; unit not printed 记录。本次发布有 11 条成绩记录,含 11 条精确值(可含历史记录),附 11 条来源定位。

成绩与评测条件

DeepSeek-V4.1-Flash · Terminal-Bench 成绩与配置
指标与成绩配置与评测条件出处
Pass@184.1原文 84.1 · 单位 未标注
公布 未确认
Codex v0.147.0
设置
Codex v0.147.0 standard app-server mode with adapted tool schemas. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
Codex v0.147.0 scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@185原文 85.0 · 单位 未标注
公布 未确认
OpenCode v1.18.15
设置
OpenCode v1.18.15 build agent with shell and file tools and native task delegation. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
OpenCode v1.18.15 scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@186.1原文 86.1 · 单位 未标注
公布 未确认
Pi v0.84.2
设置
Pi v0.84.2 RPC mode with file and shell tools plus search extension. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
Pi v0.84.2 scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@190.3原文 90.3 · 单位 未标注
公布 未确认
mini-SWE
设置
mini_swe_v2 port with a single bash tool and submission marker. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
mini-SWE
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
mini-SWE scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@185.8原文 85.8 · 单位 未标注
公布 未确认
DeepSeek Harness Standard
设置
DeepSeek Harness Standard with the full SDK profile and 26 initial tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
DeepSeek Harness Standard
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
DeepSeek Harness Standard scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@185.8原文 85.8 · 单位 未标注
公布 未确认
DeepSeek Harness PTC
设置
DeepSeek Harness PTC with run_code and 24 underlying tools. Max reasoning effort; Linux containers, temperature=1.0, top-p=0.95, 1M context and max_steps=500 as stated in the report note. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
DeepSeek Harness PTC
协议
Official DeepSeek-V4.1-Flash report Table 4; N=3 samples per task on Terminal-Bench v2.1.
条件
DeepSeek Harness PTC scaffold; Max reasoning effort. Table does not print a percent sign. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@187.3原文 87.3 · 单位 未标注
公布 未确认
Claude Code v2.1.105
设置
Claude Code v2.1.105 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 5; Max reasoning effort; row protocol from Table 4.
条件
Claude Code v2.1.105; table does not print a percent sign; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@188.4原文 88.4 · 单位 未标注
公布 未确认
Claude Code v2.1.238
设置
Claude Code v2.1.238 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 5; Max reasoning effort; row protocol from Table 4.
条件
Claude Code v2.1.238; table does not print a percent sign; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@188原文 88.0 · 单位 未标注
公布 未确认
Claude Code v2.1.251
设置
Claude Code v2.1.251 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 5; Max reasoning effort; row protocol from Table 4.
条件
Claude Code v2.1.251; table does not print a percent sign; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@187.6原文 87.6 · 单位 未标注
公布 未确认
Claude Code v2.1.259
设置
Claude Code v2.1.259 native tool interface. Max reasoning effort; same scaffold and benchmark settings as Table 4. This is an ordinary single-agent public scaffold run, not the Section 5.3.5 multi-agent experiment.
未知项
Public reproducible ordinary single-agent scaffold at explicit Max reasoning effort. Presentation uses the existing max mode; scaffold and version remain conditions and do not create a separate effort tier.
工具
允许工具
推理预算
Max
框架
未记录
协议
Official DeepSeek-V4.1-Flash report Table 5; Max reasoning effort; row protocol from Table 4.
条件
Claude Code v2.1.259; table does not print a percent sign; Max reasoning effort per the source table title. Ordinary single-agent public scaffold; not a cross-version average or multi-agent run.
在排名中查看此成绩
Pass@187.8原文 87.8 · 单位 未标注
公布 未确认
Claude Code four-version average
设置
Average computed from the unrounded Pass@1 scores of four Claude Code versions. Max reasoning effort; same scaffold and benchmark settings as Table 4.
未知项
Claude Code version-specific results are retained raw and are not mapped as ordinary effort tiers.
工具
工具条件未记录
推理预算
Max
框架
Claude Code harness
协议
Official DeepSeek-V4.1-Flash report Table 5; Max reasoning effort; row protocol from Table 4.
条件
Claude Code four-version average; table does not print a percent sign.

出处与披露

测试版本与指标

  • Pass@1 · 单位未标注 · 越高越好