Liu and colleagues: RoBERTa optimized BERT pre-training

What changed

Systematically revisited BERT training choices and reported stronger language-understanding results with longer training, more data and altered masking choices.

Scientific scope

Systematically revisited BERT training choices and reported stronger language-understanding results with longer training, more data and altered masking choices. Scope limitation: RoBERTa is a BERT pre-training replication and optimization study, not the original bidirectional-transformer invention.

Evidence

The retained primary-source abstract, bibliographic record and stated review scope support this provisional assessment. No independent experimental replication by WRS is claimed. RoBERTa is a BERT pre-training replication and optimization study, not the original bidirectional-transformer invention.

Contributors

• The named authors listed in the linked research record · SCIENCE: Complete source byline retained in the research record; human research direction, method development and evaluation. Individual labor shares are not measured.

• Liu and colleagues: RoBERTa optimized BERT pre-training — original paper
• Liu and colleagues: RoBERTa optimized BERT pre-training — official company source

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction