工具条件未记录 / max / 框架未记录 · SWE-bench Multilingual; 300 problems across nine programming languages; mean of five trials.
SWE-bench Multilingual · 300 problems, 9 programming languages · Pass rate · 工具条件未记录 / max / 框架未记录 · SWE-bench Multilingual; 300 problems across nine programming languages; mean of five trials.
Regular display scope. Rankings are grouped by metric and evaluation protocol. Recorded tool access, budgets, and frameworks are shown separately. Scores remain available for reference when conditions are unspecified.
Pass rate %
Tool access not recorded / max / Framework not recorded · SWE-bench Multilingual; 300 problems across nine programming languages; mean of five trials.
SWE-bench Multilingual · 300 problems, 9 programming languages · Pass rate · Tool access not recorded / max / Framework not recorded · SWE-bench Multilingual; 300 problems across nine programming languages; mean of five trials.
Tool access not recorded / max / Framework not recorded · SWE-bench Multilingual; 300 problems across nine programming languages; five-trial average.
SWE-bench Multilingual · 300 problems, 9 programming languages · Pass rate · Tool access not recorded / max / Framework not recorded · SWE-bench Multilingual; 300 problems across nine programming languages; five-trial average.
Tool access not recorded / max / Framework not recorded · SWE-Bench Multilingual, 300 problems across 9 programming languages; five-trial average.
SWE-bench Multilingual · 300 problems, 9 programming languages · Pass rate · Tool access not recorded / max / Framework not recorded · SWE-Bench Multilingual, 300 problems across 9 programming languages; five-trial average.
Tool access not recorded / max / Framework not recorded · 300 problems across nine programming languages; five trials averaged.
SWE-bench Multilingual · 300 problems, 9 programming languages · Pass rate · Tool access not recorded / max / Framework not recorded · 300 problems across nine programming languages; five trials averaged.