Skip to content

Source paper

NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos

Yuxue Yang, Lue Fan, Ziqi Shi and 3 more

Yuxue Yang, Lue Fan, Ziqi Shi, Junran Peng, Feng Wang, Zhaoxiang Zhang

arXiv 2601.00393 · 202689 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

This paper presents NeoVerse, a scalable 4D world model designed to reconstruct and generate novel video trajectories from in-the-wild monocular videos. The system combines a feed-forward 4D Gaussian Splatting (4DGS) reconstruction module with a video diffusion model, using a novel training strategy that simulates geometric degradation patterns to teach the generative model how to refine imperfect renderings without needing expensive multi-view ground truth.

How it works (AI summary)

The model uses a 'Gaussianized' VGGT (Vision Transformer) backbone with a bidirectional motion-encoding branch to predict 4D Gaussian parameters (position, opacity, rotation, velocity) from sparse keyframes. These Gaussians are rendered and then refined by a Video Diffusion Transformer (based on Wan-T2V), which is conditioned on simulated 'degraded renderings' created via heuristic Gaussian culling and average geometry filtering.