Modality-Dependence Reweighting and Interleaved Sparse Attention for Omni-Modal Audio-Visual-Speech Models
Generalize modality-dependence loss reweighting and interleaved sparse attention to audio-visual and streaming speech models so speculative decoding accelerates omni-modal generation.