Guu and colleagues: REALM retrieval-augmented pre-training

What changed

Pre-trained a language model with a latent document retriever whose retrieval decisions receive a masked-language-model learning signal.

Scientific scope

Pre-trained a language model with a latent document retriever whose retrieval decisions receive a masked-language-model learning signal. Scope limitation: REALM is retrieval-augmented pre-training; distinct from the later RAG generation paper already in the formal catalogue.

Evidence

The retained primary-source abstract, bibliographic record and stated review scope support this provisional assessment. No independent experimental replication by WRS is claimed. REALM is retrieval-augmented pre-training; distinct from the later RAG generation paper already in the formal catalogue.

Contributors

• The named authors listed in the linked research record · SCIENCE: Complete source byline retained in the research record; human research direction, method development and evaluation. Individual labor shares are not measured.

• Guu and colleagues: REALM retrieval-augmented pre-training — original paper
• Guu and colleagues: REALM retrieval-augmented pre-training — official company source

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction