Dynamic Multi-Step Retrieval for Long-Horizon Tasks
Empower long-horizon agents to query memory autonomously during task execution by training them to emit an active retrieval token when confidence drops.
Source paper
Zouying Cao, Jiaji Deng, Li Yu, Weikang Zhou, Zhaoyang Liu, Bolin Ding, Hai Zhao
Generated by AI. The paper’s authors did not write these.
Empower long-horizon agents to query memory autonomously during task execution by training them to emit an active retrieval token when confidence drops.
Validate extracted agent experiences objectively by running temporary sandbox simulations to verify whether candidate memories actually improve task success.
Distill high-performance summarization capabilities into a lightweight memory model by training a small architecture on outputs from a larger teacher model.
Optimize agent memory pools by clustering semantically similar experiences and using an LLM to merge them into unified canonical representations.
This paper introduces ReMe (Remember Me, Refine Me), a framework for autonomous agent evolution that replaces static memory accumulation with a dynamic lifecycle of experience extraction, reuse, and refinement. By distilling keypoint-level insights from trajectories and employing utility-based pruning, ReMe enables smaller models to outperform larger baselines on complex reasoning tasks without parameter updates.
ReMe operates in three phases: 1) Experience Acquisition, where an LLM summarizer distills success patterns and failure analysis from trajectories into structured keypoint experiences; 2) Experience Reuse, which retrieves, reranks, and rewrites experiences based on usage scenarios to guide the agent; and 3) Experience Refinement, which manages the memory pool by selectively adding valid insights and deleting low-utility entries.