BWhat changed
Zhang and Sennrich introduced RMSNorm, replacing LayerNorm mean-and-variance normalization with root-mean-square scaling, and studied partial RMS estimation as a cheaper variant.
Scientific scope
Zhang and Sennrich introduced RMSNorm, replacing LayerNorm mean-and-variance normalization with root-mean-square scaling, and studied partial RMS estimation as a cheaper variant. The transformation normalizes a stated root-mean-square statistic without subtracting the mean. Its invariance assumptions differ from layer normalization and are affected by numerical stabilization. Partial RMS estimation and full RMSNorm are distinct variants. Reported efficiency/quality results are architecture- and setup-dependent, not universal guarantees.
Evidence
Source-linked, scoped account of the original work. The published scope states the scientific assumptions and remaining verification limits. WRS has not independently certified the full proof, experimental result or system benchmark.
Contributors
- Biao Zhang · SCIENCE: Original source-listed scientific author of this scoped work; byline does not determine effort fractions.
- Rico Sennrich · SCIENCE: Original source-listed scientific author of this scoped work; byline does not determine effort fractions.
- Zhangand Sennrich, Root Mean Square Layer Normalization, exact 2019 ar Xiv 1910.07467 v 1
- Zhangand Sennrich, Root Mean Square Layer Normalization, exact 2019 ar Xiv 1910.07467 v 1 repositoryidentity
Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.
Full scientific record · Browse contribution grades · Request a correction