Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationRelated work found

Adversarial Reinforcement Learning for Scene Curriculum

Establish an adversarial feedback loop between the robot's success metrics and the scene generator to produce challenging curricula that improve policy generalization.

Time

several weeks

Compute level

Large cluster

Resources

Large compute cluster (to run massive parallel simulation for RL).

Prerequisites

Reinforcement Learning, Prompt Engineering.

Scope

modify existing code

The Gap

Section 3.5 demonstrates robot policy evaluation on generated scenes, but the flow of information is unidirectional: the scene is generated, then the robot is tested. The 'Critic' agent (Section 3.2) evaluates visual realism and layout, but lacks feedback on robot performance.

Limitation

The generator does not learn from the robot's failures. It randomly samples diversity rather than actively generating 'hard' examples that exploit specific weaknesses in the robot's policy (e.g., occlusion or reachability limits).

Proposed Solution

Close the loop by feeding the robot's success/failure metrics (from Appendix K) back into the Orchestrator's prompt refinement step. Implement an adversarial curriculum where the 'Designer' agent is rewarded for generating physically valid scenes that minimize the robot's success rate, forcing the robot policy to generalize better.

Why This Matters

This transforms the tool from a static scene generator into an active training partner, automatically discovering edge cases (like corner cases in reachability) that manual prompt engineering might miss.

How to Get Started

  1. Extract robot success and failure metrics from evaluation logs into a centralized logging format.
  2. Modify the Orchestrator prompt template to incorporate downstream robot performance feedback during scene generation.
  3. Implement an adversarial reward function for the Designer agent that penalizes high robot success rates.
  4. Run parallel simulations using Isaac Sim to measure policy generalization improvements across curriculum iterations.

Code linked to this paper on Hugging Face: github.com/nepfaff/scenesmith

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Self-Paced Prioritized Curriculum Learning and other surveys/papers discuss curriculum learning and adversarial setups in broader contexts. However, none specifically couple a scene designer agent with robot failure metrics in an adversarial curriculum to minimize robot success rates.

We searched OpenAlex for “Establish an adversarial feedback loop between the robot's success metrics and the scene generator to produce challenging curricula that improve policy generalization.” and “Adversarial Reinforcement Learning for Scene Curriculum” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

SceneSmith: Agentic Generation of Simulation-Ready Indoor Scenes

Nicholas Pfaff, Thomas Cohn, Sergey Zakharov +2 more · 2026

SceneSmith is a hierarchical, agentic framework that utilizes Large Language Models (LLMs) and Vision-Language Models (VLMs) to generate diverse, simulation-ready indoor environments from text prompts. By decomposing scene generation into sequential stages (layout, furniture, wall/ceiling objects, manipulands) managed by a Designer-Critic-Orchestrator agent trio, the system produces physically valid scenes with dense clutter and articulated objects suitable for robotic training.

Related Ideas

Is this a good idea?

Share onXLinkedIn