Interactive Policy Refinement via World Model RL
Transform a passive video world model into an interactive reinforcement learning environment so vision-language-action agents can optimize long-term task success.
Source paper
GigaBrain Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, Peng Li, Qiuping Deng, Runqi Ouyang, Wenkang Qin, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yilong Li, Yiran Ding, Yuan Xu, Yun Ye, Yukun Zhou, Zhehao Dong, Zhenan Wang, Zhichao Liu, Zheng Zhu
Generated by AI. The paper’s authors did not write these.
Transform a passive video world model into an interactive reinforcement learning environment so vision-language-action agents can optimize long-term task success.
Integrate a differentiable physics engine into diffusion-based data generation to ensure synthesized robot interactions obey fundamental physical laws.
Implement test-time model predictive control using a world model so embodied agents evaluate action consequences before execution.
This paper introduces GigaBrain-0, a Vision-Language-Action (VLA) foundation model that significantly reduces reliance on expensive real-world robot data by leveraging a "world model" (GigaWorld) to generate diverse synthetic training trajectories. The authors demonstrate that scaling data via video generation—covering variations in texture, viewpoint, and simulation-to-real transfer—combined with a model architecture incorporating RGB-D inputs and embodied Chain-of-Thought (CoT) supervision, leads to superior generalization in complex manipulation tasks.
The model utilizes a mixture-of-transformers architecture, combining a pretrained Vision-Language Model (PaliGemma2) for semantic encoding with a Diffusion Transformer (DiT) using flow matching for continuous action prediction. Training data is augmented by GigaWorld, a generative pipeline that performs Real2Real, Sim2Real, and View Transfer synthesis to create photorealistic, physically plausible variations of robot trajectories.