AWR: advantage-weighted regression for off-policy policy learning

What changed

Alternated return-target value regression with maximum-likelihood policy regression weighted exponentially by estimated advantage, enabling replay-buffer and static-dataset learning through supervised losses.

Scientific scope

Alternated return-target value regression with maximum-likelihood policy regression weighted exponentially by estimated advantage, enabling replay-buffer and static-dataset learning through supervised losses. Constrained policy-improvement motivation uses approximations including occupancy replacement. Convergent forms of supervised objectives do not prove global convergence of arbitrary neural-network SGD or universal offline policy improvement; Monte Carlo targets distinguish the original implementation from AWAC.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Constrained policy-improvement motivation uses approximations including occupancy replacement. Convergent forms of supervised objectives do not prove global convergence of arbitrary neural-network SGD or universal offline policy improvement; Monte Carlo targets distinguish the original implementation from AWAC.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction