Scaling Viewpoint Capacity via Streaming Attention in 3D RL Editing
Incorporate streaming and causal attention mechanisms into 3D multi-view editing models to decouple sequence length constraints from GPU memory limits.
Source paper
Jiyuan Wang, Chunyu Lin, Lei Sun, Zhi Cao, Yuyang Yin, Lang Nie, Zhenlong Yuan, Xiangxiang Chu, Yunchao Wei, Kang Liao, Guosheng Lin
Generated by AI. The paper’s authors did not write these.
Incorporate streaming and causal attention mechanisms into 3D multi-view editing models to decouple sequence length constraints from GPU memory limits.
Apply step-distilled diffusion backbones during the reinforcement learning exploration phase to significantly reduce inference steps and accelerate training.
Design an autoregressive batching pipeline that propagates edits across overlapping edge views to achieve consistent 360-degree scene generation.
This paper introduces RL3DEdit, a novel reinforcement learning framework for single-pass 3D scene editing that achieves high-quality, multi-view consistent results without requiring explicit 3D-paired training data. It leverages a pre-trained 3D foundation model as a verifier to guide a multi-image 2D diffusion editor, effectively anchoring 2D editing priors onto a 3D-consistent manifold.
The framework utilizes the GRPO reinforcement learning algorithm to fine-tune a multi-image 2D diffusion model (FLUX-Kontext) by exploring diverse editing candidates and evaluating their multi-view consistency using geometry-aware rewards (depth confidence, point confidence, relative pose) extracted from a frozen 3D foundation model (VGGT).