Skip to content

Source paper

SAM 3: Segment Anything with Concepts

Nicolas Carion, Laura Gustafson, Yuan-Ting Hu and 35 more

Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zhou, Liliane Momeni, Rishi Hazra, Shuangrui Ding, Sagar Vaze, Francois Porcher, Feng Li, Siyuan Li, Aishwarya Kamath, Ho Kei Cheng, Piotr Dollár, Nikhila Ravi, Kate Saenko, Pengchuan Zhang, Christoph Feichtenhofer

arXiv 2511.16719 · 2025118 upvotes on Hugging Face

Research ideas from this paper (4)

Generated by AI. The paper’s authors did not write these.

AI summary

SAM 3 introduces Promptable Concept Segmentation (PCS), a unified task for finding and segmenting all instances of a concept defined by text or image prompts in images and videos. The authors leverage a massive, AI-verified dataset and a decoupled architecture to achieve state-of-the-art results across various open-vocabulary benchmarks.

How it works (AI summary)

The model architecture consists of a DETR-based detector and a memory-aware video tracker sharing a common Hiera vision backbone, utilizing a novel 'presence token' to separate global recognition from local object queries. Its performance is driven by a scalable data engine that employs Llama 3.2-based AI verifiers to automate mask quality and exhaustivity checks.