Beltagy and colleagues: Longformer local and global sparse attention

What changed

Combined sliding-window attention and task-selected global tokens to process long documents; extended the design to encoder-decoder tasks.

Scientific scope

Combined sliding-window attention and task-selected global tokens to process long documents; extended the design to encoder-decoder tasks. Linear scaling assumes fixed local-window size and bounded global tokens. The December revision adds LED within the same report package.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Linear scaling assumes fixed local-window size and bounded global tokens. The December revision adds LED within the same report package.

Contributors

• Source-listed authors and technical contributors · SCIENCE: Source-listed authors or explicitly credited technical contributors for the cited versions. Family entries retain source-specific attribution; a combined list is not one original-paper byline or equal individual work. Names alone do not establish resolved identities.

• Longformer: The Long-Document Transformer
• Beltagy and colleagues: Longformer local and global sparse attention — author or proceedings record

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction