WRSWorld Research StateContribute
WORLD RESEARCH STATE

Retrace: clipped off-policy return traces with operator guarantees

Contribution grade

Off-policy reinforcement-learning estimators · Human-designed research method or resource

Reference score130,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Combined bounded importance-ratio traces with corrected multi-step Bellman returns, retaining long near-on-policy returns while controlling off-policy trace growth.

Scientific scope

Combined bounded importance-ratio traces with corrected multi-step Bellman returns, retaining long near-on-policy returns while controlling off-policy trace growth. Safe refers to off-policy value-operator convergence, not real-world deployment safety. Expected fixed-policy contraction uses nonnegative trace coefficients bounded by target/behavior likelihood ratios; control and stochastic convergence require additional greedy-policy, initialization, coverage and learning conditions. No guarantee for arbitrary neural approximators or absent exploration.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Safe refers to off-policy value-operator convergence, not real-world deployment safety. Expected fixed-policy contraction uses nonnegative trace coefficients bounded by target/behavior likelihood ratios; control and stochastic convergence require additional greedy-policy, initialization, coverage and learning conditions. No guarantee for arbitrary neural approximators or absent exploration.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction