WRSWorld Research StateContribute
WORLD RESEARCH STATE

MoshiVis: gated visual adaptation of speech dialogue models

Contribution grade

Multimodal speech models · Human-designed research method or resource

Reference score50,000Provisional editorial estimate of this exact contribution; its grade is assessed separately.

What changed

Augmented Moshi with lightweight cross-attention and dynamic visual gating, training adapters in one stage using mixed image-text and image-speech data to support spoken conversations about images.

Scientific scope

Augmented Moshi with lightweight cross-attention and dynamic visual gating, training adapters in one stage using mixed image-text and image-speech data to support spoken conversations about images. Inherits Moshi, visual encoders and perceptual adapters. Speechless data still uses paired image-text supervision and some speech data. Latency claims are hardware-specific; no unrestricted visual grounding or perfect preservation guarantee.

Evidence

The linked original sources describe the stated contribution and its evidence limits. WRS has not independently rerun the experiments or certified a proof. Inherits Moshi, visual encoders and perceptual adapters. Speechless data still uses paired image-text supervision and some speech data. Latency claims are hardware-specific; no unrestricted visual grounding or perfect preservation guarantee.

Contributors

Provisional WRS assessment, edition 2026-10-16.1. Independent expert calibration is pending. This grade does not certify correctness or assign ownership.

Full scientific record · Browse contribution grades · Request a correction