WRSWorld Research StateContribute
WORLD RESEARCH STATE

RedPajama: reproducible corpus mixtures and web quality signals

Contribution grade

Language-model data curation · Human-designed research method or resource

Reference score65,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Documented a best-effort open reproduction of the LLaMA1 data mixture, an unfiltered multilingual web pool with quality metadata and small-model ablations for constructing filtered subsets.

Scientific scope

Documented a best-effort open reproduction of the LLaMA1 data mixture, an unfiltered multilingual web pool with quality metadata and small-model ablations for constructing filtered subsets. The 2024 technical report documents resources released earlier. Best-effort reproduction does not provide access to the original private LLaMA corpus. The pool of more than 100 trillion tokens differs from the 50-trillion-token subset carrying quality signals. No universally best filter or blanket rights clearance is established.

Evidence

The linked original sources describe this contribution and its evidence limits. WRS has not independently reconstructed the full scientific claims. The 2024 technical report documents resources released earlier. Best-effort reproduction does not provide access to the original private LLaMA corpus. The pool of more than 100 trillion tokens differs from the 50-trillion-token subset carrying quality signals. No universally best filter or blanket rights clearance is established.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction