Skip to content

Source paper

Holi-Spatial: Evolving Video Streams into Holistic 3D Spatial Intelligence

Yuanyuan Gao, Hao Li, Yifei Liu and 14 more

Yuanyuan Gao, Hao Li, Yifei Liu, Xinhao Ji, Yuning Gong, Yuanjun Liao, Fangfu Liu, Manyuan Zhang, Yuchen Yang, Dan Xu, Xue Yang, Huaxi Huang, Hongjie Zhang, Ziwei Liu, Xiao Sun, Dingwen Zhang, Zhihang Zhong

arXiv 2603.07660 · 202669 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper introduces Holi-Spatial, a fully automated pipeline designed to convert raw video streams into high-fidelity 3D spatial annotations without human intervention. The primary contribution is the creation of Holi-Spatial-4M, a massive, spatially-aware multimodal dataset that significantly enhances the spatial reasoning and 3D grounding capabilities of Vision-Language Models.

How it works (AI summary)

The methodology consists of a three-stage framework: (1) Geometric Optimization using 3D Gaussian Splatting (3DGS) to refine monocular depth priors for multi-view consistency; (2) Image-level Perception leveraging a VLM (Gemini3-Pro) and SAM3 to generate open-vocabulary 2D segmentation masks; and (3) Scene-level Refinement that lifts 2D masks into 3D, merges redundant bounding boxes, and uses a VLM-based agent to filter and verify instances before procedurally generating spatial QA pairs.