Skip to content

Source paper

3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework

Tobias Sautter, Jan-Niklas Dihlmann, Hendrik P. A. Lensch
arXiv 2512.17459 · 20255 upvotes on Hugging Face

Research ideas from this paper (4)

Generated by AI. The paper’s authors did not write these.

Hierarchical Scene Graph Optimization for Non-Planar Placement

Extend 3D asset optimization frameworks with a hierarchical support-detection graph so child objects are positioned relative to their parent surfaces instead of the global ground plane.

3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework
🟠Medium💡Moderate InnovationSingle GPU or lessNo close match found#Generative media#Vision

Recursive Compositional Refinement for Granular Interaction

Implement a recursive decomposition loop that re-segments rendered 2D asset crops to replace coarse single-mesh compound objects with detailed discrete child meshes.

3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework
🟡Fairly easy🔧Minor TweakSingle GPU or lessNo close match found#Generative media#Vision

Iterative 3D-Guided Mask Refinement

Update target segmentation masks iteratively during 3D pose optimization by feeding back projected silhouettes to correct initial mask errors.

3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework
🟠Medium💡Moderate InnovationSingle GPU or lessRelated work found#Vision

Hybrid Asset Generation for Organic Environments

Route organic objects to specialized 3D Gaussian Splatting generators while keeping mesh generation for rigid structures via a hybrid asset generation pipeline.

3D-RE-GEN: 3D Reconstruction of Indoor Scenes with a Generative Framework
🔴Hard🔬Significant ResearchSingle GPU or lessNo close match found#Generative media#Vision

AI summary

The paper introduces 3D-RE-GEN, a pipeline for reconstructing fully textured, artist-ready 3D indoor scenes from a single 2D image. It decomposes the scene into a clean background and individual objects, ensuring physical plausibility through a novel 4-DoF ground-plane alignment optimization and using a custom visual prompting strategy called 'Application-Querying' to generate high-quality assets for occluded regions.

How it works (AI summary)

The framework integrates Grounded SAM for segmentation, Google Image Flash for context-aware inpainting (via the Application-Querying interface), and Hunyuan3D 2.0 for 3D asset generation. These components are unified by a differentiable rendering optimization loop that aligns objects to a global coordinate system using a 4-DoF 'PlanarModel' constraint, which restricts movement to the floor plane to prevent floating artifacts.