Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationRelated work found

Hierarchical Edge-Cloud Architecture for Sub-Second Reactive Replanning

Pair slow cloud frontier models with lightweight local policies for sub-second reactive parameter adaptation during dynamic robotic contact tasks.

Time

1-2 months

Compute level

Multiple GPUs

Resources

Workstation with 1-2 modern GPUs (e.g., RTX 4090 or A100), open-weight small VLM (e.g., PaliGemma or Qwen2-VL), and MuJoCo simulation.

What the paper used

Franka FR3 arm with Polymetis, robosuite 1.5.1 on MuJoCo 3.2.7, Isaac Lab 3.0.0b2, and remote frontier APIs (GPT-6 Astra, GPT-6.1 Sol, Claude Fable 5.2).

Prerequisites

Small vision-language model inference, real-time control, robot impedance control.

Scope

new model architecture

The Gap

Section 3.1 and Section J.1 note that the frontier model is queried via a remote API that incurs multi-second latency between turns. Primitives must run quasi-statically until self-termination, preventing the system from dynamically altering trajectory or force targets during execution.

“Querying a remote frontier model adds seconds of latency between primitives, which suits quasi-static contact tasks but precludes fast reactive replanning during dynamic motion.”
From the paper, Section J.1. Matched against the PDF text.

Limitation

API query latencies preclude dynamic replanning and reactive adjustments to non-stationary contact conditions during primitive execution.

Proposed Solution

Deploy a hierarchical edge-cloud architecture pairing the remote frontier model with an on-device lightweight multimodal model (e.g., 3B-7B parameters) running at 10-20 Hz. The remote model establishes high-level task plans and parameter search spaces, while the local model ingests streaming 20 Hz force-torque readings and wrist images to modulate admittance parameters (such as lead clamp L, target velocity v, or normal force Fd) mid-primitive.

Why This Matters

Overcoming the API latency bottleneck allows frozen-model robotic systems to handle dynamic and time-sensitive interactions like catching slipping objects or wiping moving surfaces.

How to Get Started

How to test it: Test on a dynamic slipping or unjamming task in MuJoCo comparing standard ForceHarness against the edge-cloud hierarchy, measuring recovery success rate and episode execution time. Stop if edge inference adds more than 100 ms loop latency or degrades admittance controller stability.

  1. Benchmark the latency profile of local 3B-7B parameter VLMs fine-tuned or prompted to predict continuous scalar control deltas from short force windows.
  2. Modify the ForceHarness inner loop in Figure A1 to accept asynchronous parameter updates from an edge process.
  3. Evaluate the hybrid controller on HiddenPhys Peg and Latch tasks subjected to time-varying external perturbations.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Papers such as IROS: A Dual-Process Architecture for Real-Time VLM-Based Indoor Navigation and Slow Brain, Fast Planner investigate hierarchical dual-process or edge-cloud configurations for latency-resilient robotics. They do not specifically target the setup of combining a remote frontier model with an on-device lightweight multimodal model ingesting 20 Hz force-torque readings for mid-primitive admittance modulation.

We searched OpenAlex and arXiv for “hierarchical VLM robot control edge cloud reactive force” and “Hierarchical Edge-Cloud Architecture for Sub-Second Reactive Replanning” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

ForceHarness: Turning Force into Action and Physics for Frozen Frontier Models

Ke Sun, Kevin Yuchen Ma, Yike Zhang +2 more · 2026

ForceHarness enables API-only, frozen frontier multimodal models to perform contact-rich robotic manipulation without demonstrations or fine-tuning. It introduces a three-layer architecture comprising force sensing and calibration, self-terminating closed-loop force primitives, and a physics translation layer that converts force traces into interpretable physical quantities. Across real-robot hardware and simulation benchmarks including HiddenPhys and ForceBench, the system dramatically improves zero-shot manipulation success and matches expert force controllers.

Related Ideas

Is this a good idea?

Share onXLinkedIn