Distributional RL and C51: categorical return learning with stability boundaries

What changed

Analyzed return-distribution Bellman operators and proposed a fixed-support categorical approximation for deep Q-learning, separating contractive policy evaluation from unstable greedy distributional control.

Scientific scope

Analyzed return-distribution Bellman operators and proposed a fixed-support categorical approximation for deep Q-learning, separating contractive policy evaluation from unstable greedy distributional control. Earlier work already studied return distributions. Under bounded-moment assumptions the fixed-policy operator contracts in maximal Wasserstein metrics; the control operator need not contract or even have a stable fixed point. These source theoretical statements are not neural C51 convergence or independently checked proofs.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Earlier work already studied return distributions. Under bounded-moment assumptions the fixed-policy operator contracts in maximal Wasserstein metrics; the control operator need not contract or even have a stable fixed point. These source theoretical statements are not neural C51 convergence or independently checked proofs.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• A Distributional Perspective on Reinforcement Learning

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction