BWhat changed
Processed attention heads in staged chunks under a context-parallel schedule and used GQA-compatible out-of-order scheduling to reduce intermediate activation memory while preserving practical training throughput.
Scientific scope
Processed attention heads in staged chunks under a context-parallel schedule and used GQA-compatible out-of-order scheduling to reduce intermediate activation memory while preserving practical training throughput. Extends Ulysses/context parallelism and uses inherited attention kernels. The87.5percent figure concerns attention intermediate tensors in selected32B settings, not total model memory. Five-million-token support uses8H100s and does not establish useful five-million-token reasoning or a universally free speed-memory tradeoff.
Evidence
The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Extends Ulysses/context parallelism and uses inherited attention kernels. The87.5percent figure concerns attention intermediate tensors in selected32B settings, not total model memory. Five-million-token support uses8H100s and does not establish useful five-million-token reasoning or a universally free speed-memory tradeoff.
Contributors
- Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.
Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.
Full scientific record · Browse contribution grades · Request a correction