Skip to content

Source paper

RealWonder: Real-Time Physical Action-Conditioned Video Generation

Wei Liu, Ziyu Chen, Zizhang Li and 3 more

Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, Jiajun Wu

arXiv 2603.05449 · 20265 upvotes on Hugging Face

Research ideas from this paper (3)

Generated by AI. The paper’s authors did not write these.

AI summary

RealWonder proposes a real-time system for generating videos conditioned on 3D physical actions (forces, gravity, collisions) from a single image. The method bridges the gap between abstract physical actions and visual generation by using a physics simulator to produce intermediate optical flow and coarse RGB previews, which then condition a distilled, fast-inference video diffusion model.

How it works (AI summary)

The pipeline consists of three stages: (1) Single-image 3D scene reconstruction using depth estimation and point clouds; (2) Real-time physics simulation (using MPM/PBD solvers) to predict dynamics and render optical flow/coarse previews; and (3) A flow-conditioned video generation model, distilled from a larger diffusion model into a 4-step causal generator for real-time streaming.