Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationNo close match found

Affordance-Grounded Execution Filters to Bridge the Text-to-Action Refusal Gap

Deploy a pre-execution safety gate that projects low-level robot commands onto a digital twin using GPU-accelerated collision checkers to bridge the text-to-action refusal gap.

Time

3-4 months (1 month for integrating cuRobo SDF and kinetic-energy checks into the harness capability contract, 1 month for perception-to-digital-twin calibration, and 1-2 months for RoboHarm, IS-Bench, and real-robot safety evaluations).

Compute level

Multiple GPUs

Resources

2x-4x RTX 4090 or A100/H100 GPUs for running cuRobo collision and SDF checks alongside frontier or open 7B-30B S2 models, an articulated robot arm with torque/force sensing and a real-time S0 controller (50 Hz-1 kHz), and high-rate calibrated workspace depth perception for dynamic digital-twin updates.

Prerequisites

Robot kinematics, trajectory generation, runtime safety verification.

Scope

Modify existing robot agent harness tool interfaces to include a middleware validation layer.

The Gap

Section 4.3 ('Safety learned in text transfers only partially to action') and Table 8 cite findings from RoboHarm showing that frontier models refuse 72% of hazardous text requests but only 18% of real-robot trials when generating direct end-effector actions, exposing a massive alignment gap between textual reasoning and physical control.

“Pooled over the same six models, refusal fell from 72% of text requests to 18% of real-robot trials with end-effector control (Sun et al., 2026a).”
From the paper, Section 4.3, page 34. Matched against the PDF text.

Limitation

Textual safety alignment (e.g., RLHF on safety dialogues) fails to reliably constrain low-level physical control tokens, allowing dangerous physical executions to bypass language safety filters.

Proposed Solution

Develop a pre-execution safety gate embedded in the harness's capability contract. Instead of checking natural language prompts, this filter projects the model's low-level end-effector or joint commands onto a geometric and dynamic digital twin of the workspace using GPU-accelerated collision checkers (e.g., cuRobo). It computes signed distance fields and kinetic energy envelopes to veto commands violating safety invariants before dispatch to S0.

Why This Matters

Ensures provable physical safety regardless of foundation model prompt jailbreaks, hallucinations, or distribution shifts during action generation.

How to Get Started

How to test it: Test the hypothesis that a GPU-accelerated SDF and kinetic-energy safety gate in the capability contract lowers serious physical safety violations without causing excessive refusal compared to text-aligned VLM baselines and unguarded JSON-schema tool interfaces. Measure serious violation rate, safe task completion rate, excessive refusal rate, and added per-call latency on RoboHarm and IS-Bench; stop if the filter adds more than 20 ms to the critical control path or triggers over 15% false-positive vetoes on benign tasks.

  1. Configure a ROS 2 workspace integrated with the cuRobo library for real-time GPU-accelerated collision checking.
  2. Construct a dynamic and geometric digital twin of the target experimental environment.
  3. Develop the safety filter module to compute signed distance fields and kinetic energy envelopes for incoming end-effector commands.
  4. Benchmark the filter using hazardous prompt scenarios from RoboHarm to measure the reduction in unsafe physical executions.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

The listed works related to text refusal are entirely sociological, literary, or high-energy physics papers, leaving the proposed geometric pre-execution safety gate and GPU-accelerated collision checking for robotics entirely untouched in this result set.

We searched OpenAlex and arXiv for “Deploy a pre-execution safety gate that projects low-level robot commands onto a digital twin using GPU-accelerated collision checkers to bridge the text-to-action refusal gap.” and “Affordance-Grounded Execution Filters to Bridge the Text-to-Action Refusal Gap” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Envisioning Agentic Robotics: A New Era of Robotic Systems

Grid Robotics, Qingfeng Lan, Duo Li +10 more · 2026

This paper presents a conceptual framework and research roadmap for 'agentic robotics', which uses foundation models within closed-loop embodied harnesses (termed Embodied System 2) to orchestrate skills, tools, and real-time control (Systems 1 and 0). The authors distinguish digital from physical agents, define a comprehensive nine-dimension evaluation protocol, and articulate a four-level ladder of delegable responsibility from basic primitives to standing autonomous roles. They argue that physical constraints—such as irreversible actions, dynamic deadlines, and unobservable contact forces—require physical agents to rebuild fundamental assumptions about verification, scaling, and safety.

Related Ideas

Is this a good idea?

Share onXLinkedIn