Process-Supervised Reasoning-Targeted Distillation for Routing-Replaced MoEs
Develop a targeted distillation pipeline combining complex reasoning datasets with step-level process reward models or Monte Carlo tree search rollouts.
Time
Compute level
Resources
What the paper used
Prerequisites
Scope
The Gap
In Section 6 and Table 2, benchmark evaluations show that performance drops significantly on complex reasoning tasks (a 6.1 point drop on AIME for the 35B tier and a 10.0 point drop for the 8B tier), whereas general tasks like MMLU-Pro and IFBench maintain parity.
“Quality loss concentrates in long-chain reasoning: 6.1 points on AIME for the 35B tier and 10.0 for the 8B tier; every other 8B benchmark is within 6.7 points, and MMLU-Pro is 4.3 points in the student’s favour.”
Limitation
The three-phase training recipe relies on general teacher-generated text and standard token-level reverse-KL distillation, failing to preserve delicate multi-step reasoning chains under approximate 4-bit quantization and altered routing paths.
Proposed Solution
Develop a targeted Phase 4 distillation pipeline specifically focused on complex reasoning datasets (such as MATH, GSM8K, and AIME). Incorporate step-level process reward model (PRM) guidance or Monte Carlo tree search (MCTS) rollouts to compute credit assignments along intermediate reasoning steps, ensuring the recovery LoRA compensates specifically for routing errors that derail deductive reasoning paths.
Why This Matters
Recovers math and long-chain logical reasoning capabilities, transforming lightweight streaming engines from conversational toys into robust tools for technical and analytical workloads.
How to Get Started
How to test it: Hypothesis: Adding a Phase 4 reasoning-targeted distillation stage using PRM/MCTS step-level credit assignment on math/deductive chains recovers at least half of the AIME 2026 gap without hurting general benchmarks. Baseline: Released Edge0-35B (86.6 AIME, 79.2 OpenCompass avg) and Edge0-8B (63.3 AIME, 69.9 avg); Metric: AIME 2026, MATH, GSM8K, and the OpenCompass 5-benchmark average; Stop if AIME 2026 gains less than 2.0 points or any non-reasoning OpenCompass benchmark drops by more than 1.5 points.
- Curate a multi-domain complex reasoning dataset combining MATH, GSM8K, and AIME problems.
- Integrate an open-source step-level Process Reward Model (PRM) or build a lightweight Monte Carlo tree search rollout harness for intermediate scoring.
- Generate step-by-step credit assignment annotations for the reasoning trajectories of routing-replaced mixture-of-experts models.
- Train recovery LoRA adapters using the PRM-weighted dataset and evaluate performance gains on AIME and MMLU-Pro benchmarks.
Code linked to this paper on Hugging Face: github.com/Edge0-AI/edge0
Suggested by AI. Adapt it to your setup.
Has This Been Done?
Works like VGPO-MCTS and SCAN explore process-supervised reward learning and step-level supervision for mathematical reasoning, while other papers address MoE self-distillation and routing behavior. However, none of these specific papers combine process reward model guidance and MCTS rollouts specifically for routing-replaced MoE distillation.
- SCAN: Self-Denoising Monte Carlo Annotation for Robust Process Reward Learning · 2025 · 11 citations
- VGPO-MCTS: Distilling Step-Level Supervision from Value-Guided Tree Search for Mathematical Reasoning · 2026 · 0 citations
- Routing Divergence Is Not Evidence of Behavioral Influence in Same-Weight MoE Self-Distillation · 2026 · 0 citations
- Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning · 2026 · 26 citations
We searched OpenAlex and arXiv for “Develop a targeted distillation pipeline combining complex reasoning datasets with step-level process reward models or Monte Carlo tree search rollouts.” and “Process-Supervised Reasoning-Targeted Distillation for Routing-Replaced MoEs” and an AI compared the idea with the top 10 results. A small search can miss relevant work.
Source Paper
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing PredictionYu Lin, Yiming Wang, Runyuan Cai +2 more · 2026
The paper introduces EDGE0, a streaming inference framework designed to serve large mixture-of-experts (MoE) models (such as 35B and 8B variants) on consumer devices constrained by limited RAM. It overcomes SSD loading latency by using a cross-token prerouter that predicts routing one token ahead, executes the prediction directly as the actual routing to completely overlap SSD expert streaming with compute, and restores accuracy loss from int4 quantization and routing replacement via an unmerged distillation LoRA.
Related Ideas
- Enhancing Abstract Visual Logic via Synthetically Augmented Multi-Step Math SFTPenguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders🟠Medium
- Distillation of Reasoning Models for Efficient Route PlanningMobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios🟡Fairly easy
- BitDistill for Reasoning-Intensive DomainsBitNet Distillation🟠Medium
- Recursive Associative-Memory Graph Traversal for Multi-Step Scientific ReasoningWeight-Space Programming Doesn't Need Transformers🟡Fairly easy
- Process-Supervised Reasoning for STEM DomainsStep-DeepResearch Technical Report🟠Medium