Learnable Neural Degradation Simulation
Replace heuristic simulation filters with a learnable degradation adapter that generates realistic rendering errors for generative refiners.
Source paper
Yuxue Yang, Lue Fan, Ziqi Shi, Junran Peng, Feng Wang, Zhaoxiang Zhang
Generated by AI. The paper’s authors did not write these.
Replace heuristic simulation filters with a learnable degradation adapter that generates realistic rendering errors for generative refiners.
Bypass 3D reconstruction for non-geometric inputs by using a depth confidence gate to fall back to video-to-video generation.
Improve motion tracking robustness by replacing hard velocity thresholds with continuous dynamic probability scores for soft temporal aggregation.
This paper presents NeoVerse, a scalable 4D world model designed to reconstruct and generate novel video trajectories from in-the-wild monocular videos. The system combines a feed-forward 4D Gaussian Splatting (4DGS) reconstruction module with a video diffusion model, using a novel training strategy that simulates geometric degradation patterns to teach the generative model how to refine imperfect renderings without needing expensive multi-view ground truth.
The model uses a 'Gaussianized' VGGT (Vision Transformer) backbone with a bidirectional motion-encoding branch to predict 4D Gaussian parameters (position, opacity, rotation, velocity) from sparse keyframes. These Gaussians are rendered and then refined by a Video Diffusion Transformer (based on Wan-T2V), which is conditioned on simulated 'degraded renderings' created via heuristic Gaussian culling and average geometry filtering.