Closed-Loop Autoregressive Training to Mitigate Prerouter Feature Drift
Implement a closed-loop training curriculum inspired by DAgger to replace ground-truth execution one-hots with model predictions during prerouter distillation.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 3.2 ('Feature drift'), the authors state that a prerouter head's input features concatenate layer hidden states with two top-k one-hot expert vectors. During training (Phase 1), these vectors correspond to the base router's ground-truth selection, but during inference, the head receives its own prior predictions because the true router is replaced entirely.
“At training that is the base router’s selection; at decode it is the head’s own prediction, because the prediction replaced the router.”
Limitation
The prerouter heads are trained under teacher-forcing conditions and are never exposed to their own routing errors during distillation, creating exposure bias and compounding prediction errors across sequential tokens.
Proposed Solution
Implement a closed-loop training curriculum inspired by DAgger or Scheduled Sampling for routing prediction. During Phase 1 distillation, gradually replace ground-truth execution one-hots with the model's own predicted top-k vectors with increasing probability over training steps. Furthermore, add a multi-step rollout loss that penalizes multi-token divergent trajectories, stabilizing prerouter representations against out-of-distribution drift.
Why This Matters
Mitigating compounding routing prediction drift improves routing fidelity over long generation horizons, reducing reliance on the recovery LoRA and narrowing the accuracy gap on long-context tasks.
How to Get Started
How to test it: Hypothesis: Closed-loop scheduled sampling and multi-step rollout loss during Phase 1 prerouter training reduce routing exposure bias and improve downstream accuracy. Baseline: Edge0-35B and Edge0-8B trained with standard teacher-forced Phase 1 followed by Phases 2-3; Metric: OpenCompass 5-benchmark average, AIME 2026 score, and multi-token prerouter agreement; Stop if after full Phase 1-3 training the OpenCompass average improves by less than 0.5 points over the baseline.
- Extract the Phase 1 distillation dataset containing layer hidden states and ground-truth expert routing vectors from an open-source teacher model like Qwen3.6-35B.
- Implement a scheduled sampling loop that progressively replaces ground-truth routing one-hots with the student model's own top-k predictions.
- Define a multi-step rollout loss function to penalize divergence in sequential token prediction paths.
- Evaluate prerouter accuracy and downstream task performance on single-node 8x A100/H100 setups against the standard non-adaptive baseline.
Code linked to this paper on Hugging Face: github.com/Edge0-AI/edge0
Suggested by AI. Adapt it to your setup.
Has This Been Done?
None of the listed papers discuss autoregressive training curricula, DAgger, or Scheduled Sampling for routing prediction, as all retrieved results pertain to high-energy physics and astrophysics experiments.
We searched OpenAlex and arXiv for “Implement a closed-loop training curriculum inspired by DAgger to replace ground-truth execution one-hots with model predictions during prerouter distillation.” and “Closed-Loop Autoregressive Training to Mitigate Prerouter Feature Drift” and an AI compared the idea with the top 5 results. A small search can miss relevant work.
Source Paper
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing PredictionYu Lin, Yiming Wang, Runyuan Cai +2 more · 2026
The paper introduces EDGE0, a streaming inference framework designed to serve large mixture-of-experts (MoE) models (such as 35B and 8B variants) on consumer devices constrained by limited RAM. It overcomes SSD loading latency by using a cross-token prerouter that predicts routing one token ahead, executes the prediction directly as the actual routing to completely overlap SSD expert streaming with compute, and restores accuracy loss from int4 quantization and routing replacement via an unmerged distillation LoRA.
Related Ideas
- Distillation of Reasoning Models for Efficient Route PlanningMobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios🟡Fairly easy
- Accelerating Inference in Spatial-TTT via Predictive Fast-Weight UpdatesSpatial-TTT: Streaming Visual-based Spatial Intelligence with Test-Time Training🔴Hard
- Stabilizing On-Policy Distillation via Critic RegularizationLiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation🟡Fairly easy
- Optimizing Agent Performance via Iterative Reinforcement Learning and Data FilteringOpenSeeker: Democratizing Frontier Search Agents by Fully Open-Sourcing Training Data🔴Hard
- Self-Supervised Skill Distillation for Oracle-Free EvolutionSkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning🟡Fairly easy