Natural Questions: real search queries with full-page answer annotation

What changed

Paired anonymized Google search queries with Wikipedia pages, annotating long evidence regions and short answers or null answers, with repeated annotations and evaluation designed to account for human variability.

Scientific scope

Paired anonymized Google search queries with Wikipedia pages, annotating long evidence regions and short answers or null answers, with repeated annotations and evaluation designed to account for human variability. Original task supplies the full page and includes answerability decisions, unlike later NQ-Open derivatives. Annotation ambiguity is explicitly studied rather than removed by a single definitive label. Byline symbols describe specific roles and affiliations, not a blanket equal-contribution claim.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Original task supplies the full page and includes answerability decisions, unlike later NQ-Open derivatives. Annotation ambiguity is explicitly studied rather than removed by a single definitive label. Byline symbols describe specific roles and affiliations, not a blanket equal-contribution claim.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Natural Questions: A Benchmark for Question Answering Research

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction