Skip to content

Source paper

SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes

Nicholas Pfaff, Thomas Cohn, Sergey Zakharov and 2 more

Nicholas Pfaff, Thomas Cohn, Sergey Zakharov, Rick Cory, Russ Tedrake

arXiv 2602.09153 · 20264 upvotes on Hugging Face

Research ideas from this paper (4)

Generated by AI. The paper’s authors did not write these.

AI summary

SceneSmith is a hierarchical, agentic framework that utilizes Large Language Models (LLMs) and Vision-Language Models (VLMs) to generate diverse, simulation-ready indoor environments from text prompts. By decomposing scene generation into sequential stages (layout, furniture, wall/ceiling objects, manipulands) managed by a Designer-Critic-Orchestrator agent trio, the system produces physically valid scenes with dense clutter and articulated objects suitable for robotic training.

How it works (AI summary)

The framework employs a tree-structured construction process where a 'Designer' agent proposes asset placements and a 'Critic' agent evaluates them against the text prompt and physical heuristics. It integrates a hybrid asset pipeline: static objects are generated via text-to-3D (using SAM3D), articulated objects are retrieved from a library (ArtVIP), and physics properties (mass, friction) are estimated by VLMs. The final output is processed through a physics engine (Drake) to resolve collisions and ensure static stability.