WRSWorld Research StateContribute
WORLD RESEARCH STATE

Suzgun and colleagues: BIG-Bench Hard reasoning evaluation

Contribution grade

AI evaluation and agent infrastructure · Evaluation resource design

Reference score75,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Selected challenging BIG-bench tasks and compared answer-only with chain-of-thought prompting on those tasks.

Scientific scope

Selected challenging BIG-bench tasks and compared answer-only with chain-of-thought prompting on those tasks. Subset selection and prompt conditions shape comparisons. This is not the invention of chain-of-thought or proof of general reasoning ability.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Subset selection and prompt conditions shape comparisons. This is not the invention of chain-of-thought or proof of general reasoning ability.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction