GPT-f: transformer-guided formal proof search and shorter library proofs

What changed

GPT-f combines transformer-generated Metamath proof steps with guided backward search, reporting56.22% held-out theorem reproduction at32 search attempts per theorem and23 shortened proofs contributed to set.mm.

Scientific scope

GPT-f combines transformer-generated Metamath proof steps with guided backward search, reporting56.22% held-out theorem reproduction at32 search attempts per theorem and23 shortened proofs contributed to set.mm. The assessed target is arXiv 2009.03393v1. The reported 56.22% result is held-out test performance under its stated proof-search budget; 56.50% is a different validation result. The 23 contributed proofs shorten proofs of existing library theorems and are not 23 newly discovered mathematical theorems. Metamath checking depends on the chosen library statements and hypotheses. Human model development, existing library authorship and model proof execution retain separate credit. WRS has not independently trained or replayed the system.

Evidence

Source-linked, scoped account of the original work. The published scope states the scientific assumptions and remaining verification limits. WRS has not independently certified the full proof, experimental result or system benchmark.

Contributors

• Stanislas Polu and Ilya Sutskever (complete original byline) · SCIENCE: Scientific system design, training, experiments and paper; no individual effort fractions inferred.
• GPT-f / OpenAI (source-reported model) · EXECUTION: Automated proof-step generation/search within reported benchmark and library-shortening scope; not a human author.
• Metamath community and formal verification tooling · VERIFICATION: Human curation, discussion, library maintenance and source-reported formal checks. WRS has not independently rerun these checks.

• Polu and Sutskever 2020: Generative Language Modeling for Automated Theorem Proving — ar Xiv 2009.03393 v 1
• Metamath set.mm PR 1547: Shortening of various proofs based on Open AI provers
• Metamath set.mm PR 1561: Open AI prover shortening of supmax, dprdwd, unitg, bnj 538

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction