Gemini API model id gemini-3-pro-preview; default sampling settings. Gemini results are pass@1 and single-attempt unless specified; smaller benchmarks average multiple trials.
未知项
The evaluation table does not specify a thinking budget or inference-time compute setting.
工具
工具条件未记录
推理预算
未记录
框架
Sierra framework
协议
Retail category; standard Sierra framework.
条件
Methodology notes Gemini is used as the user model with a system instruction.
Gemini API model id gemini-3.1-pro-preview; default sampling settings; evaluation table column specifies Thinking (High). All scores are pass@1 except where noted.
未知项
Exact thinking budget and sampling parameters are not listed.
工具
工具条件未记录
推理预算
high
框架
Sierra framework
协议
Retail category; standard Sierra framework; prompt adjustment for environment instructions; user model uses Gemini with a system instruction.
条件
Airline category excluded from the report due to lower-quality grading.
Differences are calculated only for unique, active, exact values between direct predecessors when the unit, metric, and explicitly recorded configuration and conditions match.
· 1 score record for this release, including 1 exact value (may include historical records)
Gemini 3 Pro · τ²-bench scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Success rate85.3%Reported 85.3% · Unit %
Published Unconfirmed
Gemini 3 Pro API default
Settings
Gemini API model id gemini-3-pro-preview; default sampling settings. Gemini results are pass@1 and single-attempt unless specified; smaller benchmarks average multiple trials.
Unknowns
The evaluation table does not specify a thinking budget or inference-time compute setting.
Tools
Tool access not recorded
Reasoning budget
not recorded
Framework
Sierra framework
Protocol
Retail category; standard Sierra framework.
Conditions
Methodology notes Gemini is used as the user model with a system instruction.
· 1 score record for this release, including 1 exact value (may include historical records)
Records for the direct predecessor do not provide matching conditions for a valid difference.
Gemini 3.1 Pro · τ²-bench scores and configurations
Metric and score
Configuration and evaluation conditions
Sources
Success rate90.8%Reported 90.8% · Unit %
Published Unconfirmed
Gemini 3.1 Pro Thinking (High)
Settings
Gemini API model id gemini-3.1-pro-preview; default sampling settings; evaluation table column specifies Thinking (High). All scores are pass@1 except where noted.
Unknowns
Exact thinking budget and sampling parameters are not listed.
Tools
Tool access not recorded
Reasoning budget
high
Framework
Sierra framework
Protocol
Retail category; standard Sierra framework; prompt adjustment for environment instructions; user model uses Gemini with a system instruction.
Conditions
Airline category excluded from the report due to lower-quality grading.