Radford and colleagues: generative pre-training for language understanding (GPT)

What changed

Combined transformer language-model pre-training on unlabeled text with supervised fine-tuning and reported transfer improvements across language-understanding tasks.

Scientific scope

Combined transformer language-model pre-training on unlabeled text with supervised fine-tuning and reported transfer improvements across language-understanding tasks. Scope limitation: This is the 2018 first GPT study, not the invention of transformers or pre-training generally. The reported results are specific to the paper's task suite.

Evidence

The retained primary-source abstract, bibliographic record and stated review scope support this provisional assessment. No independent experimental replication by WRS is claimed. This is the 2018 first GPT study, not the invention of transformers or pre-training generally. The reported results are specific to the paper's task suite.

Contributors

• The named authors listed in the linked research record · SCIENCE: Complete source byline retained in the research record; human research direction, method development and evaluation. Individual labor shares are not measured.

• Radford and colleagues: generative pre-training for language understanding (GPT) — original paper
• Radford and colleagues: generative pre-training for language understanding (GPT) — company research source

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction