Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization Horizons
Retrieve relevant harness failure histories dynamically using semantic embedding search so recursive self-improvement scales across long continual adaptation sequences without overflowing context limits.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
Section 3.2 describes memory ingestion: 'At each round, the proposer receives the complete SIM in its system prompt, and the selector reads the same record when assessing candidates. Both read all entries without retrieval or truncation.'
“At each round, the proposer receives the complete SIM in its system prompt, and the selector reads the same record when assessing candidates. Both read all entries without retrieval or truncation.”
Limitation
Passing the entire un-truncated memory into prompts restricts CRSI to short single-step transitions of 10 rounds; extended continual learning sequences across many datasets or model checkpoints will cause context overflow, high token consumption, and attention degradation.
Proposed Solution
Construct a hierarchical vector-indexed memory system for SIM. Embed root-cause problem hypotheses and edit rationales using a dense code-and-text embedding model. At each proposal and selection round, condition retrieval on the active evolve-set failure trace embeddings to fetch the top-k most relevant historical attempts, while summarizing older inactive problem clusters into compact regression constraints.
The paper’s authors list this direction as future work, so they may already be pursuing it.
Why This Matters
Enabling efficient memory retrieval would allow agent harnesses to self-improve continually across dozens of task distributions and model iterations without context bloat or memory dilution.
How to Get Started
How to test it: Evaluate across a 5-benchmark sequential transition chain. Compare hierarchical retrieval against naive full-context concatenation and sliding-window FIFO truncation on Terminal-Bench and text classification suites; measure final task score, token overhead per round, and stopping when retrieval fails to maintain within 1 point of full concatenation.
- Clone the public Terminus-2 and AHE harnesses along with the Terminal-Bench evaluation harness.
- Implement a JSON vector store wrapper around SIM entries using open-source embeddings to index problem definitions and modification rationales.
- Modify the proposer prompt generator to substitute full-history dumping with top-3 retrieved problem records based on rollout error trace similarity.
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Papers such as A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces and Self-Harness: Harnesses That Improve Themselves touch upon agentic memory and self-improving harnesses separately. However, none of the listed results combine a hierarchical vector-indexed memory system conditioned on active failure trace embeddings specifically for extended optimization horizons in self-improvement loops.
- Self-Harness: Harnesses That Improve Themselves · 2026 · 148 citations
- A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces · 2026 · 41 citations
We searched OpenAlex and arXiv for “continual agent memory retrieval self-improvement harness” and “Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization Horizons” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
Self-Improvement Memory Enables Continual Recursive Self-ImprovementXin Yu, Fei Xia, Jiaqi Pan +7 more · 2026
This paper introduces Continual Recursive Self-Improvement (CRSI), a framework for continuing LLM agent harness optimization across changes in task benchmarks or underlying policy models. To prevent loss of context and redundant edits across transitions, the authors propose Self-Improvement Memory (SIM), which records harness diagnoses, attempted code modifications, rationales, and verified outcomes. Across coding, text classification, agentic workspace, and engineering benchmarks, maintaining SIM enables higher downstream performance and more effective reuse and revision of inherited harness components compared to harness-only transfer.
Related Ideas
- Importance-Weighted Semantic Memory Compaction for Ultra-Long Horizon TasksGamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent🟡Fairly easy
- Dynamic Multi-Step Retrieval for Long-Horizon TasksRemember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution🟠Medium
- Learnable Temporal-Semantic Affinity for ConsolidationSimpleMem: Efficient Lifelong Memory for LLM Agents🟠Medium
- Dynamic Working-Memory Pruning for Search Agents to Overcome Reinforcement Learning SaturationK2-Horizon: A Frontier Fleet of Open Source Language Models🟡Fairly easy
- Dynamic Hierarchical Layout Optimization for Token-Level MemoryMemory in the Age of AI Agents🔴Hard