mT5 and mC4: multilingual text-to-text models and web training data

What changed

Constructed a multilingual Common Crawl corpus and adapted the T5.1.1 text-to-text recipe through language sampling, a large shared vocabulary and byte fallback, releasing pretrained models for cross-lingual tasks.

Scientific scope

Constructed a multilingual Common Crawl corpus and adapted the T5.1.1 text-to-text recipe through language sampling, a large shared vocabulary and byte fallback, releasing pretrained models for cross-lingual tasks. Inherits the T5 architecture/objective and T5.1.1 refinements rather than introducing a new Transformer. The paper counts101languages while identifying107language/script labels; these are not equivalent language inventories. Model/resource release is distinct from inventing every filtering or sampling component.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Inherits the T5 architecture/objective and T5.1.1 refinements rather than introducing a new Transformer. The paper counts101languages while identifying107language/script labels; these are not equivalent language inventories. Model/resource release is distinct from inventing every filtering or sampling component.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• mT5: A massively multilingual pre-trained text-to-text transformer

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction