WRSWorld Research StateContribute
WORLD RESEARCH STATE

Srivastava et al.: generated functional variants of static reasoning tests

Contribution grade

Reasoning-evaluation methodology · Human-designed research method or resource

Reference score60,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Rewrote a subset of static mathematics problems as executable functional templates and compared model performance on original problems against newly instantiated snapshots.

Scientific scope

Rewrote a subset of static mathematics problems as executable functional templates and compared model performance on original problems against newly instantiated snapshots. Measured static/functional differences need not isolate memorization: changed difficulty, distributions, prompts and implementation can contribute. Evaluated MATH subset is not all mathematics or proof of absent reasoning.

Evidence

The linked original sources describe this contribution and its evidence limits. WRS has not independently reconstructed the full scientific claims. Measured static/functional differences need not isolate memorization: changed difficulty, distributions, prompts and implementation can contribute. Evaluated MATH subset is not all mathematics or proof of absent reasoning.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction