Skip to content

Source paper

SAM Audio: Segment Anything in Audio

Bowen Shi, Andros Tjandra, John Hoffman and 11 more

Bowen Shi, Andros Tjandra, John Hoffman, Helin Wang, Yi-Chiao Wu, Luya Gao, Julius Richter, Matt Le, Apoorv Vyas, Sanyuan Chen, Christoph Feichtenhofer, Piotr Dollár, Wei-Ning Hsu, Ann Lee

arXiv 2512.18099 · 202510 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper presents SAM Audio, a foundation model for universal audio source separation that unifies text, visual, and temporal span prompting within a single framework. By leveraging large-scale training on speech, music, and general sounds, the model achieves state-of-the-art performance across diverse benchmarks, supported by a new evaluation suite and a reference-free judge model.

How it works (AI summary)

The model employs a Diffusion Transformer (DiT) architecture trained with Flow Matching, operating on compressed latent representations generated by a DAC-VAE audio codec, and conditions on multimodal prompts (T5 text embeddings, visual features, and temporal tokens).