WRSWorld Research StateContribute
WORLD RESEARCH STATE

Cohere: decentralized expert-track post-training and model merging

Contribution grade

Language-model training systems · Human-designed research method or resource

Reference score65,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Alternated centralized instruction tuning with separately optimized capability experts, parameter merging and polishing to integrate multilingual, retrieval and tool-use capabilities in Command A; documented an interleaved local/global attention implementation.

Scientific scope

Alternated centralized instruction tuning with separately optimized capability experts, parameter merging and polishing to integrate multilingual, retrieval and tool-use capabilities in Command A; documented an interleaved local/global attention implementation. Integration and training recipe, not invention of model soups, GQA, RoPE, NoPE, CoPG or SRPO. Expert tracks are merged parameter sets rather than token-routed MoE. Weights are non-commercial research releases; throughput and benchmark claims are not independently certified.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Integration and training recipe, not invention of model soups, GQA, RoPE, NoPE, CoPG or SRPO. Expert tracks are merged parameter sets rather than token-routed MoE. Weights are non-commercial research releases; throughput and benchmark claims are not independently certified.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction