WRSWorld Research StateContribute
WORLD RESEARCH STATE

AReaL: asynchronous rollouts with staleness-aware policy updates

Contribution grade

Distributed language-model reinforcement learning · Human-designed research method or resource

Reference score75,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Decoupled continuous generation from training, using interruptible rollouts, bounded data staleness, dynamic batching and a decoupled PPO objective for trajectories generated across policy versions.

Scientific scope

Decoupled continuous generation from training, using interruptible rollouts, bounded data staleness, dynamic batching and a decoupled PPO objective for trajectories generated across policy versions. Inherits asynchronous RL and decoupled PPO. Weight updates can interrupt generation, discard/recompute KV caches and create mixed-policy trajectories; staleness must be constrained, not ignored. Up to2.57fold acceleration and matched accuracy concern selected math/code experiments, not universal convergence or arbitrary off-policy stability.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Inherits asynchronous RL and decoupled PPO. Weight updates can interrupt generation, discard/recompute KV caches and create mixed-policy trajectories; staleness must be constrained, not ignored. Up to2.57fold acceleration and matched accuracy concern selected math/code experiments, not universal convergence or arbitrary off-policy stability.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction