LARS: layerwise rate scaling for large-batch image training

What changed

Scaled per-layer learning rates using weight and gradient norms to train image classifiers at much larger batch sizes while studying accuracy and hardware efficiency.

Scientific scope

Scaled per-layer learning rates using weight and gradient norms to train image classifiers at much larger batch sizes while studying accuracy and hardware efficiency. Inherits data parallelism, warmup and SGD. ResNet/AlexNet results do not guarantee unchanged generalization at arbitrary batch sizes, datasets or architectures. Parallel-hardware and schedule choices are part of the reported speedup.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Inherits data parallelism, warmup and SGD. ResNet/AlexNet results do not guarantee unchanged generalization at arbitrary batch sizes, datasets or architectures. Parallel-hardware and schedule choices are part of the reported speedup.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Scaling SGD Batch Size to 32K for ImageNet Training

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction