Skip to content
AI-generated research idea🟡Fairly easy💡Moderate InnovationRelated work found

Importance-Weighted Semantic Memory Compaction for Ultra-Long Horizon Tasks

Implement an adaptive, importance-scored memory hierarchy using semantic retrieval and pinned global states to prevent critical loss in ultra-long horizon tasks.

Time

3 to 5 weeks (1 week to implement the pinned register, cross-attention eviction, and RoPE cache updates in Streaming-vLLM, 1-2 weeks for SFT and TA-GRPO retraining, and 1-2 weeks for 3-trial side-quest and 50-hour main-story evaluations).

Compute level

Large cluster

Resources

A distributed A800 cluster for SFT of Qwen3-VL-8B on the ~1K-hour multi-level HSR dataset (~390M tokens) with the modified memory manager, plus a Megatron/MCore trainer and a 64 NVIDIA H200 Streaming-vLLM pool serving 32 parallel Windows game VMs for ~14.6K TA-GRPO rollouts and multi-hour evaluations.

What the paper used

Controlled CPT scaling used 120 configurations across Qwen3.5 Dense (0.8B, 2B, 9B, 27B) and ~45K NVIDIA H200 GPU-hours. The main Gamer model (Qwen3-VL-8B; also Qwen3.5-35B-A3B MoE in CPT) trained on ~2.4K hours gameplay plus ~6.3B VLM tokens on a distributed H200 cluster (CPT), ~1K hours HSR annotations (~706K samples) on a distributed A800 cluster (SFT), and ~14.6K online rollouts using Megatron/MCore with 64 NVIDIA H200 GPUs serving 32 parallel Windows VMs (RL).

Prerequisites

Long-context transformers, retrieval-augmented generation (RAG), memory models.

Scope

Modify context formatting and eviction routines in Verl and Streaming-vLLM without altering underlying model weights.

The Gap

In Section 3.2 and Section 5.1, Gamer manages long-term context by updating a memory summary buffer capped at 512 tokens. When the token budget is reached, the system enforces a strict first-in, first-out (FIFO) eviction rule where the oldest complete summaries are permanently removed from the conditioning prompt.

“When the memory budget is exceeded, the oldest complete summaries are removed in temporal order.”
From the paper, Section 3.2, page 8. Matched against the PDF text.

Limitation

FIFO eviction causes critical early-episode context (such as overarching quest objectives, persistent item states, or structural environmental constraints) to be permanently discarded during multi-stage quests exceeding 30-100 minutes.

Proposed Solution

Replace the FIFO queue with an adaptive, importance-scored memory hierarchy. Implement a two-tiered memory manager: (1) a pinned global-state register retaining immutable quest goals and key inventory states, and (2) a semantic retrieval module that computes cross-attention relevance between candidate historical summaries and the current task state, evicting low-relevance intermediate summaries rather than strictly older ones.

Why This Matters

Prevents catastrophic memory loss over extended multi-objective workflows, allowing agents to solve complex questlines involving frequent backtracking and delayed prerequisite resolution.

How to Get Started

How to test it: Test whether replacing FIFO eviction (512-token budget) with a pinned global-state register and cross-attention semantic retrieval improves ultra-long-horizon state retention by comparing against the FIFO Gamer-RL baseline on Opportunity Makes the Thief (Part I) (30 min), Bestial Ferocity (100 min), and the 50-hour main-storyline evaluation, measuring milestone accuracy and autonomous completion rate. Stop if overall accuracy on Bestial Ferocity does not surpass Gamer-RL's 83.6% or if semantic retrieval pushes Streaming-vLLM request latency above the 200 ms step budget.

  1. Collect offline trajectory datasets featuring long-horizon interaction tasks and episodic memory constraints.
  2. Design the pinned global-state register for storing immutable quest goals and key inventory states.
  3. Implement a cross-attention relevance scoring module to evaluate historical summaries against current task states.
  4. Compare the semantic retrieval memory manager against standard FIFO queue eviction baselines on long-horizon success metrics.

Code linked to this paper on Hugging Face: github.com/RainBowLuoCS/Gamer

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Imprint: Online Memory Compression for Long-Horizon Egocentric QA and Fine-Mem: Fine-Grained Feedback Alignment for Long-Horizon Memory Management explore advanced memory compression and management for long-horizon tasks, but do not implement the specific two-tiered pinned global-state register combined with cross-attention relevance retrieval proposed here.

We searched OpenAlex and arXiv for “Implement an adaptive, importance-scored memory hierarchy using semantic retrieval and pinned global states to prevent critical loss in ultra-long horizon tasks.” and “Importance-Weighted Semantic Memory Compaction for Ultra-Long Horizon Tasks” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent

Run Luo, Kuan Zhang, Qi Deng +24 more · 2026

This paper presents Gamer, a generalist Vision-Language-Action (VLA) foundation model designed for real-time, long-horizon interactive 3D and 2D gaming environments using raw screen observations and standard keyboard-mouse actions. The system employs a three-stage training pipeline comprising continued pre-training (CPT) with trajectory debiasing, multi-level supervised fine-tuning (SFT) with sparse reasoning and memory compaction, and Temporal-Aware Group Relative Policy Optimization (TA-GRPO) using game-native verifiers. To achieve sub-100ms latency necessary for real-time closed-loop control and rollout generation, the authors develop Streaming-vLLM, featuring stateful KV-cache reuse, incremental RoPE cache updates, and FSM-guided speculative decoding.

Related Ideas

Is this a good idea?

Share onXLinkedIn