Hoffmann and colleagues: compute-optimal language-model training and Chinchilla

What changed

Estimated a compute-optimal balance of model parameters and training tokens, then tested it with the 70-billion-parameter Chinchilla model trained on substantially more tokens than the earlier Gopher model at a comparable training-compute budget.

Scientific scope

Estimated a compute-optimal balance of model parameters and training tokens, then tested it with the 70-billion-parameter Chinchilla model trained on substantially more tokens than the earlier Gopher model at a comparable training-compute budget. Scope limitation: The equal scaling of model size and data is an empirical result under the paper's transformer, data and compute assumptions, not a universal theorem or proof that Chinchilla dominates every later model.

Evidence

The retained primary-source abstract, bibliographic record and stated review scope support this provisional assessment. No independent experimental replication by WRS is claimed. The equal scaling of model size and data is an empirical result under the paper's transformer, data and compute assumptions, not a universal theorem or proof that Chinchilla dominates every later model.

Contributors

• The named authors listed in the linked research record · SCIENCE: Complete source byline retained in the research record; human research direction, method development and evaluation. Individual labor shares are not measured.

• Training Compute-Optimal Large Language Models
• An empirical analysis of compute-optimal large language model training — Google DeepMind

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction