WRSWorld Research StateContribute
WORLD RESEARCH STATE

RULER: configurable tests beyond single-needle retrieval

Contribution grade

Long-context evaluation · Evaluation resource design

Reference score75,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Built synthetic tasks with controllable sequence length and complexity, extending needle retrieval to multiple items, multi-hop tracing, aggregation and question answering.

Scientific scope

Built synthetic tasks with controllable sequence length and complexity, extending needle retrieval to multiple items, multi-hop tracing, aggregation and question answering. Reported effective context sizes depend on these tasks and the chosen success threshold. Synthetic proxy performance is not a universal usable-context guarantee or a measure of every long-document skill. Original first two authors share contribution.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Reported effective context sizes depend on these tasks and the chosen success threshold. Synthetic proxy performance is not a universal usable-context guarantee or a measure of every long-document skill. Original first two authors share contribution.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction