WRSWorld Research StateContribute
WORLD RESEARCH STATE

The Pile: documented mixture of diverse language-model corpora

Contribution grade

Language-model training resources · Human-designed research method or resource

Reference score145,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Assembled a documented825GiB English text mixture from22sources, released processing code and studied cross-domain language-model training and evaluation.

Scientific scope

Assembled a documented825GiB English text mixture from22sources, released processing code and studied cross-domain language-model training and evaluation. The arXiv identifier begins2101 but original submission is31December2020, retained as source date. General language-model resource, not a new discovery in constituent subject domains; domain-specific biomedical subsets were not separately acquired or curated. Inherits constituent corpora; mixing weights/effective exposures differ from unique raw data. No universal generalization, current availability or content/rights clearance.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. The arXiv identifier begins2101 but original submission is31December2020, retained as source date. General language-model resource, not a new discovery in constituent subject domains; domain-specific biomedical subsets were not separately acquired or curated. Inherits constituent corpora; mixing weights/effective exposures differ from unique raw data. No universal generalization, current availability or content/rights clearance.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction