Claude Opus · Legal Agent Benchmark · Opus 5 官方发布表所列 held-out set · 历代记录

按发布时间查看 Claude Opus 的 3 次发布在 Legal Agent Benchmark · Opus 5 官方发布表所列 held-out set 的成绩、配置与披露。

仅为直接前代间的唯一、有效精确值计算差值,要求单位、指标与明确记录的配置和条件一致。

Claude Opus 4.8

· 本次发布有 2 条成绩记录,含 2 条精确值(可含历史记录)

Claude Opus 4.8 · Legal Agent Benchmark 成绩与配置
指标与成绩配置与评测条件出处
All-pass rate9.62%原文 9.62% · 单位 %
公布 未确认
Legal Agent Benchmark held-out
设置
Adaptive thinking/max effort;1,235 problems,16/1,251 个问题因数据缺陷在测试前排除;n=5 trials 平均;Sonnet 4.6 judge;内部重实现只提供 bash 与 Python,少于公开 harness 的工具集。
未知项
未另记
工具
允许工具
推理预算
max
框架
未记录
协议
Legal Agent Benchmark held-out problem set; standard all-criteria-pass task success.
条件
1,235 problems tested; 16/1,251 removed for data defects before testing; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge.
在排名中查看此成绩
Mean criterion-pass rate89.01%原文 89.01% · 单位 %
公布 未确认
Legal Agent Benchmark held-out
设置
Adaptive thinking/max effort;1,235 problems,16/1,251 个问题因数据缺陷在测试前排除;n=5 trials 平均;Sonnet 4.6 judge;内部重实现只提供 bash 与 Python,少于公开 harness 的工具集。
未知项
未另记
工具
允许工具
推理预算
max
框架
未记录
协议
Legal Agent Benchmark held-out problem set; mean rubric-criterion pass rate.
条件
1,235 problems tested; average over 5 trials; internal harness uses bash and Python only with Sonnet 4.6 judge. Metric is a proposed extension to the held-out catalog entry.
在排名中查看此成绩

Claude Opus 5

· 本次发布有 2 条成绩记录,含 2 条精确值(可含历史记录)

与直接前代的记录缺少满足差值计算的同条件依据。

Claude Opus 5 · Legal Agent Benchmark 成绩与配置
指标与成绩配置与评测条件出处
All-pass rate11.7%原文 11.7% · 单位 %
公布 未确认
Legal Agent Benchmark · Harvey held-out set
设置
External Harvey evaluation on its held-out set; effort, API, toolset, and run count not specified in the system card.
未知项
Do not combine with Anthropic internal full-set evaluation.
工具
工具条件未记录
推理预算
未记录
框架
未记录
协议
Legal Agent Benchmark; Harvey held-out set; all-pass rate.
条件
External Harvey evaluation; effort/API/tool details not stated; separate from the Anthropic full-set result.
在排名中查看此成绩
Mean criterion-pass rate94.1%原文 94.1% · 单位 %
公布 未确认
Legal Agent Benchmark · Harvey held-out set
设置
External Harvey evaluation on its held-out set; effort, API, toolset, and run count not specified in the system card.
未知项
Do not combine with Anthropic internal full-set evaluation.
工具
工具条件未记录
推理预算
未记录
框架
未记录
协议
Legal Agent Benchmark; Harvey held-out set; mean criterion-pass rate.
条件
External Harvey evaluation; effort/API/tool details not stated; separate from the Anthropic full-set result.
在排名中查看此成绩

Claude Opus 5.5

· 本次发布有 2 条成绩记录,含 2 条精确值(可含历史记录)

与直接前代的记录缺少满足差值计算的同条件依据。

Claude Opus 5.5 · Legal Agent Benchmark 成绩与配置
指标与成绩配置与评测条件出处
All-pass rate8.3%原文 8.3% all-pass rate · 单位 %
公布 未确认
System Card max effort (per-result benchmark harnesses)
设置
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
未知项
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Harvey LAB held-out 120-task set; max effort; Artificial Analysis evaluation harness.
条件
Distinct from the public full LAB set; do not label as Harvey's public leaderboard score. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.
在排名中查看此成绩
Mean criterion-pass rate91.2%原文 91.2% mean criterion-pass rate · 单位 %
公布 未确认
System Card max effort (per-result benchmark harnesses)
设置
System Card §8.1 default: adaptive thinking/max effort and default sampling settings unless a section says otherwise. Benchmark tools, evaluator, safeguards, fallback, and repetitions remain per-result conditions; this config represents the stated model effort, not harness equivalence.
未知项
Exact run time is not disclosed.、Per-benchmark tools, safeguards, and repetition counts are specified only where the source states them.
工具
工具条件未记录
推理预算
max
框架
未记录
协议
Harvey LAB held-out 120-task set; max effort; Artificial Analysis evaluation harness.
条件
Criterion-pass rate is not the all-pass rate; keep the two metrics separate. Source-wide conditions: Harvey LAB 120-problem held-out set; independently reported by Artificial Analysis on its own harness; distinct from the public 2,010+ task set.
在排名中查看此成绩