LeCun, Denker and Solla: Optimal Brain Damage

What changed

A trained neural network may contain weights that contribute little to its predictions. LeCun, Denker and Solla estimate the cost of removing a weight from how sharply the training error bends near the learned solution. They remove low-cost weights and retrain. Their experiments show why curvature can be more useful than simply keeping large weights, while also showing that a local approximation becomes less accurate when too many weights are removed at once.

Scientific scope

A trained neural network may contain weights that contribute little to its predictions. LeCun, Denker and Solla estimate the cost of removing a weight from how sharply the training error bends near the learned solution. They remove low-cost weights and retrain. Their experiments show why curvature can be more useful than simply keeping large weights, while also showing that a local approximation becomes less accurate when too many weights are removed at once. The NIPS contribution indexed by its publisher under 1989. The saliency estimate uses a locally quadratic loss, near-stationary parameters and a diagonal-Hessian approximation. Reported pruning and retraining results are finite experiments; the name does not assert globally optimal pruning. Network redesign and automatic weight deletion are separate sources of compression.

Evidence

The linked original works support the scoped statement and contribution attribution. WRS has not independently replayed experiments or reconstructed a full mathematical proof. The NIPS contribution indexed by its publisher under 1989. The saliency estimate uses a locally quadratic loss, near-stationary parameters and a diagonal-Hessian approximation. Reported pruning and retraining results are finite experiments; the name does not assert globally optimal pruning. Network redesign and automatic weight deletion are separate sources of compression.

Contributors

• The named authors in the research record · SCIENCE: Source-listed authors for the exact contribution; experimental subjects are not autonomous scientific authors. Source support and predecessor work retain separate roles. Individual effort shares and unresolved identities are not inferred.

• Official NeurIPS record for Optimal Brain Damage
• Yann LeCun publication record for Optimal Brain Damage
• NeurIPS publisher record for the 1989 contribution

Provisional WRS assessment, edition 2026-10-23.3. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction