TL;DR
Weakening-Induced On-Policy Self-Distillation (WISD) is a novel framework that allows Large Reasoning Models (LRMs) to improve their own performance without needing any external labels, reference solutions, or expert feedback. By contrasting a model with a "weakened" version of itself—created through simple, training-free techniques like pruning or quantization—the model identifies and reinforces its own successful reasoning patterns, achieving significant accuracy gains across math and coding benchmarks.
What Problem Does This Paper Address?
To improve reasoning, models often use On-Policy Distillation (OPD), where a "student" model learns from a "teacher" model by analyzing the student's own generated solution paths. Traditionally, this requires a stronger teacher, which is usually constructed by giving the model privileged information—such as the correct answer, an expert demonstration, or feedback from a code execution environment.
However, high-quality labeled data is expensive and scarce. The authors ask a provocative question: Can a model improve its reasoning through self-distillation without any privileged information? WISD proves that you don't need a "smarter" teacher to learn; you just need a "dumber" one to serve as a point of contrast.
Methods
The core innovation of WISD is reversing the standard teacher-student dynamic. Instead of building a stronger teacher with extra context, the authors take the original model ($\pi_0$) as the "stronger" member and create a weakened copy ($\pi_w$) using two training-free weight degradation methods:
- Low-bit Quantization: Coarsening the model's weights (e.g., down to 3-bit precision).
- Magnitude Pruning: Removing the smallest weights (e.g., zeroing out the bottom 1/6th of weights).
As shown in Figure 2, the original model generates reasoning trajectories on unlabeled prompts. Both the original and weakened models then "score" these trajectories. The token-level contrast—the difference in log-probabilities between the original and the weakened model—serves as the training signal. The student is trained to favor tokens where the original model is significantly more confident than its weakened counterpart, effectively "distilling" the reasoning capabilities that are lost during weight degradation.
Datasets
The authors evaluated WISD using 3,200 unlabeled prompts per domain:
- Training: Math prompts from Skywork-OR1-RL-Data and coding prompts from KlearReasoner-CodeSub-15K.
- Evaluation (Math): AIME 2024, AIME 2025, and HMMT 2025 (Harvard-MIT Math Tournament).
- Evaluation (Code): LiveCodeBench (LCB) versions 5 and 6.
The study spanned diverse architectures, including dense models (Qwen3, Qwen3.5, OLMo-3) and Mixture-of-Experts (MoE) models (Qwen3.6-35B-A3B).
Key Insights and Ideas
- The "Goldilocks" Zone of Weakening: There is a non-monotonic relationship between how much you weaken a model and how much it learns. If the weakening is too mild (e.g., 8-bit quantization), there isn't enough contrast. If it's too aggressive (e.g., 2-bit), the weakened model becomes too noisy to provide a useful signal. The authors found that 3-bit quantization and a 1/6th pruning threshold were the "sweet spots."
- Focusing on "Hard" Tokens: Analysis showed that WISD naturally allocates more "supervision budget" to critical reasoning roles. Specifically, it places higher training weight on exploration (proposing tentative steps) and verification (checking intermediate results) tokens compared to their natural frequency in the data.
- Implicit Error Correction: By contrasting with a version of itself that is prone to "forgetting" subtle logic or numerical precision, the model learns to double-down on the exact steps where its reasoning is most robust but fragile.
Key Results
WISD achieved consistent, state-of-the-art improvements across the board, often outperforming methods that do use privileged information.
- Average Gains: Quantization and pruning improved average accuracy by 4.0 and 5.5 percentage points, respectively.
- Math Performance: On the challenging HMMT 2025 benchmark, pruning raised Qwen3.5-4B’s accuracy from 57.7% to 70.5%.
- Coding Performance: On LiveCodeBench v6, the same model jumped from 39.1% to 56.7%.
- Scalability: The method works for large-scale MoE models too; Qwen3.6-35B-A3B saw its LCB v6 accuracy rise from 61.8% to 69.5% via quantization.
Limitations and Future Work
- Base Capability Requirement: WISD is a self-improvement framework. It likely requires a base model that already possesses a baseline level of reasoning ability to generate viable trajectories.
- Hyperparameter Tuning: The effectiveness of the method depends on finding the right level of weight degradation, which might vary for different model families.
- Future Directions: The authors suggest exploring more sophisticated weakening operators beyond simple pruning/quantization and investigating how WISD could be combined with verifiable reward signals (like unit tests in coding) to further boost performance.
Summary updated