Skip to content
AI-generated research idea🟠Medium🔬Significant ResearchRelated work found

Process-Supervised Reasoning-Targeted Distillation for Routing-Replaced MoEs

Develop a targeted distillation pipeline combining complex reasoning datasets with step-level process reward models or Monte Carlo tree search rollouts.

Time

2-3 weeks to generate and score PRM/MCTS reasoning rollouts, run Phase 4 LoRA distillation for 8B and 35B, and complete OpenCompass evaluations.

Compute level

Multiple GPUs

Resources

A multi-GPU compute cluster (e.g., 4x-8x A100/H100 80GB) to run fp16 35B/8B teacher MCTS/PRM rollouts and Phase 4 recovery LoRA fine-tuning, plus OpenCompass evaluation compute.

What the paper used

Training runs on dequantized bf16 35B and 8B models over ~2M Phase 2 SFT rows and ~200k Phase 3 on-policy rows (training GPUs not stated); quality is measured with OpenCompass on a compute server against the fp16 base models.

Prerequisites

Reinforcement learning from AI feedback (RLAIF), process reward models, knowledge distillation.

Scope

Modify existing code and add custom distillation objective pipelines.

The Gap

In Section 6 and Table 2, benchmark evaluations show that performance drops significantly on complex reasoning tasks (a 6.1 point drop on AIME for the 35B tier and a 10.0 point drop for the 8B tier), whereas general tasks like MMLU-Pro and IFBench maintain parity.

“Quality loss concentrates in long-chain reasoning: 6.1 points on AIME for the 35B tier and 10.0 for the 8B tier; every other 8B benchmark is within 6.7 points, and MMLU-Pro is 4.3 points in the student’s favour.”
From the paper, Section 6, page 11. Matched against the PDF text.

Limitation

The three-phase training recipe relies on general teacher-generated text and standard token-level reverse-KL distillation, failing to preserve delicate multi-step reasoning chains under approximate 4-bit quantization and altered routing paths.

Proposed Solution

Develop a targeted Phase 4 distillation pipeline specifically focused on complex reasoning datasets (such as MATH, GSM8K, and AIME). Incorporate step-level process reward model (PRM) guidance or Monte Carlo tree search (MCTS) rollouts to compute credit assignments along intermediate reasoning steps, ensuring the recovery LoRA compensates specifically for routing errors that derail deductive reasoning paths.

Why This Matters

Recovers math and long-chain logical reasoning capabilities, transforming lightweight streaming engines from conversational toys into robust tools for technical and analytical workloads.

How to Get Started

How to test it: Hypothesis: Adding a Phase 4 reasoning-targeted distillation stage using PRM/MCTS step-level credit assignment on math/deductive chains recovers at least half of the AIME 2026 gap without hurting general benchmarks. Baseline: Released Edge0-35B (86.6 AIME, 79.2 OpenCompass avg) and Edge0-8B (63.3 AIME, 69.9 avg); Metric: AIME 2026, MATH, GSM8K, and the OpenCompass 5-benchmark average; Stop if AIME 2026 gains less than 2.0 points or any non-reasoning OpenCompass benchmark drops by more than 1.5 points.

  1. Curate a multi-domain complex reasoning dataset combining MATH, GSM8K, and AIME problems.
  2. Integrate an open-source step-level Process Reward Model (PRM) or build a lightweight Monte Carlo tree search rollout harness for intermediate scoring.
  3. Generate step-by-step credit assignment annotations for the reasoning trajectories of routing-replaced mixture-of-experts models.
  4. Train recovery LoRA adapters using the PRM-weighted dataset and evaluate performance gains on AIME and MMLU-Pro benchmarks.

Code linked to this paper on Hugging Face: github.com/Edge0-AI/edge0

Suggested by AI. Adapt it to your setup.

Has This Been Done?

Related work found

Works like VGPO-MCTS and SCAN explore process-supervised reward learning and step-level supervision for mathematical reasoning, while other papers address MoE self-distillation and routing behavior. However, none of these specific papers combine process reward model guidance and MCTS rollouts specifically for routing-replaced MoE distillation.

We searched OpenAlex and arXiv for “Develop a targeted distillation pipeline combining complex reasoning datasets with step-level process reward models or Monte Carlo tree search rollouts.” and “Process-Supervised Reasoning-Targeted Distillation for Routing-Replaced MoEs” and an AI compared the idea with the top 10 results. A small search can miss relevant work.

Source Paper

The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction

Yu Lin, Yiming Wang, Runyuan Cai +2 more · 2026

The paper introduces EDGE0, a streaming inference framework designed to serve large mixture-of-experts (MoE) models (such as 35B and 8B variants) on consumer devices constrained by limited RAM. It overcomes SSD loading latency by using a cross-token prerouter that predicts routing one token ahead, executes the prediction directly as the actual routing to completely overlap SSD expert streaming with compute, and restores accuracy loss from int4 quantization and routing replacement via an unmerged distillation LoRA.

Related Ideas

Is this a good idea?

Share onXLinkedIn