Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationRelated work found

Dynamic Multi-Step Retrieval for Long-Horizon Tasks

Empower long-horizon agents to query memory autonomously during task execution by training them to emit an active retrieval token when confidence drops.

Time

3-4 weeks

Compute level

Large cluster

Resources

A multi-GPU cluster for fine-tuning the agent policy model and maintaining an active Elasticsearch vector database during generation trajectories.

What the paper used

Qwen3-Instruct models (8B, 14B, 32B), text-embedding-v4 with dimension 1024, evaluated on BFCL-V3 and AppWorld benchmarks.

Prerequisites

Agentic workflows, reinforcement learning or advanced prompt engineering.

Scope

modify existing code

The Gap

Currently, ReMe employs a fixed retrieval strategy where experiences are fetched only once at the beginning of a task (Section 3.3). However, benchmarks like AppWorld involve long execution horizons where the task context and sub-goals shift significantly after initial tool interactions.

“First, ReMe currently employs a fixed retrieval strategy, where experiences are retrieved once at the beginning of each task.”
From the paper, Limitations, page 9. Matched against the PDF text.

Limitation

The single-turn retrieval mechanism fails to provide relevant guidance for intermediate steps in multi-turn tasks. If an agent encounters a specific error or sub-problem halfway through a trajectory, the experiences retrieved based on the initial query are likely irrelevant, leading to 'blind' trial-and-error in later stages.

Proposed Solution

Implement an 'Active Retrieval' mechanism where the agent can autonomously trigger memory lookups during execution. This involves fine-tuning the agent to emit a special token (e.g., <RETRIEVE>) when confidence is low or an error occurs, using the current conversation history and tool output as the query to fetch context-relevant experiences dynamically.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

This would significantly improve success rates on complex, multi-step benchmarks like AppWorld by ensuring the agent has access to the right procedural knowledge exactly when it is needed, rather than relying solely on initial context.

How to Get Started

How to test it: We hypothesize that active retrieval improves intermediate step success over static initial retrieval. Baseline: ReMe with fixed single-turn retrieval. Metric: task success rate on multi-turn subsets. Stopping condition: failure to exceed baseline task success by at least 2% after active intervention fine-tuning.

  1. Define a special retrieval token and construct fine-tuning data with simulated low-confidence triggers in AppWorld or BFCL.
  2. Modify the agent decoding loop to intercept the retrieval token and execute dynamic memory queries using the conversation history.
  3. Fine-tune the agent policy on mixed standard and retrieval-augmented execution traces.
  4. Evaluate the active retrieval agent on multi-step benchmarks to measure error recovery and task success rates.

Code linked to this paper on Hugging Face: github.com/agentscope-ai/ReMe

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Works like 'ActiveMem: Dynamic Latent Memory Trees for Long-Horizon Agents' and 'STaR: Scalable Task-Conditioned Retrieval for Long-Horizon Multi-Modal Robot Memory' touch upon dynamic retrieval mechanisms and active memory management for agents. However, the specific token-triggered active retrieval mechanism using conversation history and tool outputs leaves room for unique implementation details not fully resolved by the listed papers.

We searched OpenAlex for “Empower long-horizon agents to query memory autonomously during task execution by training them to emit an active retrieval token when confidence drops.” and “Dynamic Multi-Step Retrieval for Long-Horizon Tasks” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution

Zouying Cao, Jiaji Deng, Li Yu +4 more · 2025

This paper introduces ReMe (Remember Me, Refine Me), a framework for autonomous agent evolution that replaces static memory accumulation with a dynamic lifecycle of experience extraction, reuse, and refinement. By distilling keypoint-level insights from trajectories and employing utility-based pruning, ReMe enables smaller models to outperform larger baselines on complex reasoning tasks without parameter updates.

Related Ideas

Is this a good idea?

Share onXLinkedIn