Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution

Yue He, Jiahui Hu, Shaman Tang, Hua Zhou, Qiman Wu, Menglin Li, Yizhao Wang, Yuheng Zhang, Da Zhu, Zhou Wang, Anhua Liu, Hanlin Chen, Mingyuan Wang, Rui Xin, Yuhua Wei, Shuang Su, Siqi Liu, Kai Yang, Weichao Huang, Hang Zhang, Honggou Yang, Xianming Liu

TL;DR

The researchers at XPeng Inc. have developed a framework to consolidate multiple domain-specialist LLMs into a single, high-performance dynamic Mixture-of-Experts (MoE) model. By introducing Calibrated Multi-teacher On-Policy Distillation (C-MOPD) and an optimized execution path, they achieved near-perfect integration of specialist capabilities while speeding up the training process by 2.86×.

What Problem Does This Paper Address?

In modern LLM post-training, it is common to develop "specialists"—models fine-tuned via Reinforcement Learning (RL) to excel in specific domains like coding or mathematics. However, deploying a fleet of separate specialists is computationally expensive and difficult to manage.

The challenge lies in capability integration: how do you merge these specialists into one student model without losing their unique strengths? Previous methods often suffered from "exposure mismatch" (where the student sees data differently than the teacher did) or "loss scale imbalance" (where one teacher's loss function dominates others during training). Furthermore, training dynamic MoE models is notoriously slow due to the overhead of routing tokens to different experts and the serial nature of generating data (rollouts) before updating the model.

Methods

The authors propose a three-stage pipeline: Shared SFT $\rightarrow$ Domain-Specific RL $\rightarrow$ C-MOPD Consolidation.

Figure 1: The Turing LLM post-training framework. Shared supervised initialization provides the starting point for domain-specialist RL. C-MOPD consolidates the frozen specialists into one student through teacher-local Top-K matching and loss-scale calibration on student-generated prefixes. Bounded-width dispatch, route replay following R3, and asynchronous overlap support efficient dynamic-MoE execution in the consolidation workflow.

1. Calibrated Multi-teacher On-Policy Distillation (C-MOPD)

To merge specialists, the student model generates its own responses (on-policy), and a task router selects the appropriate specialist to act as a teacher. The student then tries to match the teacher's probability distribution for the top tokens.

  • Loss-Scale Calibration: Since a "coding" loss might have a different magnitude than a "math" loss, the authors apply teacher-wise normalization.
  • Token-Level Clipping: This prevents outlier tokens from producing massive gradients that could destabilize training.

2. Efficient Dynamic-MoE Execution

To solve the training bottleneck, they implemented several systems-level optimizations:

  • RW32 (Bounded-width dispatch): Standard MoE models reserve many "slots" for experts. By capping the dispatch width to 32 experts per token, they reduced the memory footprint by 8× with negligible impact on accuracy.
  • Final-Route Replay (R3): This ensures that the expert routing chosen during the data generation phase is exactly reused during the training update phase, maintaining consistency.
  • AsyncOverlap: They overlapped the "rollout" (generation) and "actor" (training) phases. While the trainer updates the model using batch $i$, the rollout engine generates batch $i+1$ using a slightly older version of the policy (one-step policy lag).
Execution optimizations including RW32 and AsyncOverlap.

Datasets

  • SFT Stage: 5.27M examples covering math, code, tool use, and dialogue.
  • Consolidation Pool: 51,500 high-quality prompts specifically curated for Math, Code, Instruction Following (IF), and Agent tasks.
  • Evaluation: A massive suite including MMLU, CMMLU, C-Eval, GSM8K, MATH, HumanEval, MBPP, IFEval, and BFCL v4 (Berkeley Function-Calling Leaderboard).

Key Insights and Ideas

  • Calibration is the Secret Sauce: The "aha moment" here is that raw distillation losses are not comparable across domains. Without normalization and clipping, the student model's performance significantly degrades because it over-optimizes for the "loudest" teacher.
  • MoE Efficiency is a Systems Problem: The 2.86× speedup wasn't achieved through a better optimizer, but through clever scheduling (AsyncOverlap) and reducing redundant expert slots (RW32).
  • Dual-Mode Support: The model (Turing-20B-A2B) natively supports both "Think" mode (Chain-of-Thought reasoning) and direct response, controlled by simple prefix markers like <think>.

Key Results

  • Integration Success: The consolidated model achieved a normalized integration score of 0.9988, meaning it successfully captured nearly 100% of the gains made by the individual specialists.
  • State-of-the-Art Reasoning: In reasoning-enabled benchmarks (with an 8K response cap), Turing-20B-A2B outperformed Qwen3-8B on all four major math and code benchmarks, achieving an unweighted mean of 85.92 vs. Qwen's 80.33.
  • Training Speed: The optimized execution path reduced the time per training step from 398.7 seconds to 139.3 seconds.

Limitations and Future Work

  • GPQA Performance: The authors noted a slight decrease (5.05 points) in the GPQA Diamond benchmark compared to the SFT baseline, suggesting that while reasoning improved, some niche scientific knowledge might have been "washed out" during consolidation.
  • Policy Lag: While AsyncOverlap provides a massive speedup, it introduces a "one-step policy lag." While this didn't hurt performance in this study, the long-term stability of this approach for even larger models or longer training runs remains an open question.
  • Future Directions: The framework currently uses fixed teacher weights. Future work could explore dynamic weighting where the student learns which teacher to trust more based on real-time performance metrics.

Summary updated

Abstract

We study post-training for dynamically routed mixture-of-experts (MoE) models, focusing on integrating independently optimized specialists while keeping consolidation efficient. Our approach combines calibrated multi-teacher on-policy distillation (C-MOPD), which matches each selected teacher on a local Top-$K$ support with teacher-wise loss normalization and clipping, with an execution path that uses post-capacity bounded-width dispatch, final-route replay, and asynchronous rollout--training overlap. On Turing-20B-A2B, the selected model achieves a four-domain normalized integration score of 0.9988 and retains at least 90.16\% of the measured specialist gain in every domain. At matched $K=32$, the full calibration objective scores 0.9962, compared with 0.9724 without calibration and 0.9834 without clipping. At a matched 8K response cap, the model exceeds both shared SFT and Qwen3-8B on all four reasoning-enabled mathematics and code benchmarks (unweighted mean: 85.92, versus 80.33 for Qwen3-8B and 74.21 for SFT), and improves over SFT on ten of twelve broader evaluation metrics. In a PPU-based profile, the combined execution configuration reduces consolidation step time from 398.7 to 139.3 seconds, an observed 2.86$\times$ speedup over serial execution. These results demonstrate strong specialist integration alongside more efficient consolidation, while reflecting the complete training and execution framework rather than any single component.
BibTeX
@misc{he2026turing,
  author = {Yue He and Jiahui Hu and Shaman Tang and Hua Zhou and Qiman Wu and Menglin Li and Yizhao Wang and Yuheng Zhang and Da Zhu and Zhou Wang and Anhua Liu and Hanlin Chen and Mingyuan Wang and Rui Xin and Yuhua Wei and Shuang Su and Siqi Liu and Kai Yang and Weichao Huang and Hang Zhang and Honggou Yang and Xianming Liu},
  title = {{Turing LLM Post-Training: Calibrated Distillation and Efficient Dynamic-MoE Execution}},
  year = {2026},
  eprint = {2610.39876},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39876},
}
Figure