WRSWorld Research StateContribute
WORLD RESEARCH STATE

Sparse experts: more stored capacity without activating every expert

Contribution grade

Sparse neural architectures · Sparse neural architectures

Reference score140,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

The 2017 sparsely-gated mixture-of-experts layer routes each input to a small subset of learned networks, coupling large stored capacity to selective computation and explicit load-balancing mechanisms.

Scientific scope

The 2017 sparsely-gated mixture-of-experts layer routes each input to a small subset of learned networks, coupling large stored capacity to selective computation and explicit load-balancing mechanisms. Sparse gating activates a subset of experts in the specified neural architectures. Capacity, load balancing, communication and training overhead remain costs; parameter count is not compute used per example. Reported language-model/translation results are specific benchmarks, not a universal performance guarantee. Later routing and mixture-of-experts systems retain separate credit.

Evidence

Source-linked, scoped account of the original work. The published scope states the scientific assumptions and remaining verification limits. WRS has not independently certified the full proof, experimental result or system benchmark.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction