Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationNo close match found

Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization Horizons

Retrieve relevant harness failure histories dynamically using semantic embedding search so recursive self-improvement scales across long continual adaptation sequences without overflowing context limits.

Time

several weeks

Compute level

Single GPU or less

Resources

Standard GPU for local embedding generation or access to an embedding API, and public benchmarks (Terminal-Bench 2.1 and 4).

What the paper used

Gemini 3.8 Flash and Claude Opus 5.5 accessed via API; coding rounds take approximately 4 hours each (1.7h evaluation, 34m candidate sampling) for N=10 rounds with k=5 candidates per round.

Prerequisites

embedding models, vector search, prompt engineering, agent harness design

Scope

modify existing code

The Gap

Section 3.2 describes memory ingestion: 'At each round, the proposer receives the complete SIM in its system prompt, and the selector reads the same record when assessing candidates. Both read all entries without retrieval or truncation.'

“At each round, the proposer receives the complete SIM in its system prompt, and the selector reads the same record when assessing candidates. Both read all entries without retrieval or truncation.”
From the paper, Section 3.2, page 4. Matched against the PDF text.

Limitation

Passing the entire un-truncated memory into prompts restricts CRSI to short single-step transitions of 10 rounds; extended continual learning sequences across many datasets or model checkpoints will cause context overflow, high token consumption, and attention degradation.

Proposed Solution

Construct a hierarchical vector-indexed memory system for SIM. Embed root-cause problem hypotheses and edit rationales using a dense code-and-text embedding model. At each proposal and selection round, condition retrieval on the active evolve-set failure trace embeddings to fetch the top-k most relevant historical attempts, while summarizing older inactive problem clusters into compact regression constraints.

The paper’s authors list this direction as future work, so they may already be pursuing it.

Why This Matters

Enabling efficient memory retrieval would allow agent harnesses to self-improve continually across dozens of task distributions and model iterations without context bloat or memory dilution.

How to Get Started

How to test it: Evaluate across a 5-benchmark sequential transition chain. Compare hierarchical retrieval against naive full-context concatenation and sliding-window FIFO truncation on Terminal-Bench and text classification suites; measure final task score, token overhead per round, and stopping when retrieval fails to maintain within 1 point of full concatenation.

  1. Clone the public Terminus-2 and AHE harnesses along with the Terminal-Bench evaluation harness.
  2. Implement a JSON vector store wrapper around SIM entries using open-source embeddings to index problem definitions and modification rationales.
  3. Modify the proposer prompt generator to substitute full-history dumping with top-3 retrieved problem records based on rollout error trace similarity.

Suggested by AI. Adapt it to your setup.

Has This Been Done?

No close match in 10 results

Papers such as A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces and Self-Harness: Harnesses That Improve Themselves touch upon agentic memory and self-improving harnesses separately. However, none of the listed results combine a hierarchical vector-indexed memory system conditioned on active failure trace embeddings specifically for extended optimization horizons in self-improvement loops.

We searched OpenAlex and arXiv for “continual agent memory retrieval self-improvement harness” and “Hierarchical Retrieval-Augmented Self-Improvement Memory for Extended Optimization Horizons” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Self-Improvement Memory Enables Continual Recursive Self-Improvement

Xin Yu, Fei Xia, Jiaqi Pan +7 more · 2026

This paper introduces Continual Recursive Self-Improvement (CRSI), a framework for continuing LLM agent harness optimization across changes in task benchmarks or underlying policy models. To prevent loss of context and redundant edits across transitions, the authors propose Self-Improvement Memory (SIM), which records harness diagnoses, attempted code modifications, rationales, and verified outcomes. Across coding, text classification, agentic workspace, and engineering benchmarks, maintaining SIM enables higher downstream performance and more effective reuse and revision of inherited harness components compared to harness-only transfer.

Related Ideas

Is this a good idea?

Share onXLinkedIn