Skip to content
AI-generated research idea🟠Medium💡Moderate InnovationClosely matching work found

Dynamic Step-Wise Skill Retrieval

Perform dynamic, step-wise skill retrieval indexed by both task descriptions and current environment observations to provide just-in-time reasoning support.

Time

30 hours

Compute level

Large cluster

Resources

8 NVIDIA H100 80GB GPUs for 30 hours

What the paper used

Qwen2.5-7B-Instruct with 8 NVIDIA H100 80GB GPUs

Prerequisites

transformer architectures

Scope

modify existing code

The Gap

Equation (4) in Section 3.2 defines skill retrieval based solely on the task description embedding (ed): Sret = TopK(sim(ed, es)). This means the set of retrieved skills is static for the entire episode.

“At inference, given a task description d, the agent retrieves relevant skills to augment its context.”
From the paper, Section 3.2, page 4. Matched against the PDF text.

Limitation

In multi-step tasks (e.g., ALFWorld), the skills required change dynamically (e.g., 'Navigation' skills are needed first, 'Manipulation' skills later). Static retrieval fills the context window with skills that may be relevant to the *task* but irrelevant to the *current state*, wasting tokens and potentially confusing the agent.

Proposed Solution

Modify the inference loop to perform dynamic retrieval at regular intervals or upon sub-goal completion. The retrieval query should be a function of both the task description and the current observation/history (ed oplus eot), ensuring that the agent has access to 'Just-In-Time' skills relevant to the immediate bottleneck.

Why This Matters

This would significantly improve token efficiency and reasoning accuracy in long-horizon tasks by reducing context noise and providing precise, state-relevant guidance.

How to Get Started

How to test it: We hypothesize that dynamic step-wise skill retrieval improves multi-step execution over static task-level retrieval. The baseline is static retrieval, measured by trajectory token efficiency and ALFWorld success rate, with a 5% improvement required to continue.

  1. Modify the inference loop to query the SkillBank at regular time intervals and sub-goal boundaries.
  2. Construct a joint retrieval query embedding combining task description vectors and current observation vectors.
  3. Index existing skill libraries using vector databases like FAISS with multi-modal embedding support.
  4. Evaluate task success rates on long-horizon benchmarks like Mind2Web comparing static versus dynamic retrieval.

Code linked to this paper on Hugging Face: github.com/aiming-lab/SkillRL

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Closely matching work found

The paper 'Just-in-Time Skill Retrieval via Long-Horizon State Tracking' directly proposes dynamic skill retrieval based on state tracking and task descriptions. Other works like 'VISOR: Agentic Visual Retrieval-Augmented Generation via Iterative Search and Over-horizon Reasoning' explore iterative retrieval, but the first result essentially executes the proposed dynamic step-wise skill retrieval framework.

We searched OpenAlex for “Perform dynamic, step-wise skill retrieval indexed by both task descriptions and current environment observations to provide just-in-time reasoning support.” and “Dynamic Step-Wise Skill Retrieval” and an AI compared the idea with the top 6 results. A small search can miss relevant work.

Source Paper

SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning

Peng Xia, Jianwen Chen, Hanyang Wang +10 more · 2026

The paper introduces SKILLRL, a framework that enables LLM agents to learn from past experiences by distilling successful and failed trajectories into a hierarchical skill library (General and Task-Specific) using a teacher model. This library is then used to augment the agent's context during Reinforcement Learning (specifically GRPO), where the skill library recursively evolves based on validation failure modes to continuously improve performance.

Related Ideas

Is this a good idea?

Share onXLinkedIn