Weaken to Strengthen: Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models

Yecheng Wu, Song Han, Han CaiarXiv 2026 · arXiv

TL;DR

Weakening-Induced On-Policy Self-Distillation (WISD) is a novel framework that allows Large Reasoning Models (LRMs) to improve their own performance without needing any external labels, reference solutions, or expert feedback. By contrasting a model with a "weakened" version of itself—created through simple, training-free techniques like pruning or quantization—the model identifies and reinforces its own successful reasoning patterns, achieving significant accuracy gains across math and coding benchmarks.

What Problem Does This Paper Address?

To improve reasoning, models often use On-Policy Distillation (OPD), where a "student" model learns from a "teacher" model by analyzing the student's own generated solution paths. Traditionally, this requires a stronger teacher, which is usually constructed by giving the model privileged information—such as the correct answer, an expert demonstration, or feedback from a code execution environment.

However, high-quality labeled data is expensive and scarce. The authors ask a provocative question: Can a model improve its reasoning through self-distillation without any privileged information? WISD proves that you don't need a "smarter" teacher to learn; you just need a "dumber" one to serve as a point of contrast.

Methods

The core innovation of WISD is reversing the standard teacher-student dynamic. Instead of building a stronger teacher with extra context, the authors take the original model ($\pi_0$) as the "stronger" member and create a weakened copy ($\pi_w$) using two training-free weight degradation methods:

  1. Low-bit Quantization: Coarsening the model's weights (e.g., down to 3-bit precision).
  2. Magnitude Pruning: Removing the smallest weights (e.g., zeroing out the bottom 1/6th of weights).
Figure 2. Overview of WISD. (a) Prior self-distillation. Privileged information c, such as solutions, demonstrations, or feedback, strengthens the original model into a self-teacher. (b) WISD. We instead retain the original model as the stronger member and construct a weaker copy through training-free quantization or pruning. The frozen pair scores the same self-generated trajectories, and its token-level log-probability contrast provides cached supervision for a student initialized from the original model, without privileged information.

As shown in Figure 2, the original model generates reasoning trajectories on unlabeled prompts. Both the original and weakened models then "score" these trajectories. The token-level contrast—the difference in log-probabilities between the original and the weakened model—serves as the training signal. The student is trained to favor tokens where the original model is significantly more confident than its weakened counterpart, effectively "distilling" the reasoning capabilities that are lost during weight degradation.

Datasets

The authors evaluated WISD using 3,200 unlabeled prompts per domain:

  • Training: Math prompts from Skywork-OR1-RL-Data and coding prompts from KlearReasoner-CodeSub-15K.
  • Evaluation (Math): AIME 2024, AIME 2025, and HMMT 2025 (Harvard-MIT Math Tournament).
  • Evaluation (Code): LiveCodeBench (LCB) versions 5 and 6.

The study spanned diverse architectures, including dense models (Qwen3, Qwen3.5, OLMo-3) and Mixture-of-Experts (MoE) models (Qwen3.6-35B-A3B).

Key Insights and Ideas

  • The "Goldilocks" Zone of Weakening: There is a non-monotonic relationship between how much you weaken a model and how much it learns. If the weakening is too mild (e.g., 8-bit quantization), there isn't enough contrast. If it's too aggressive (e.g., 2-bit), the weakened model becomes too noisy to provide a useful signal. The authors found that 3-bit quantization and a 1/6th pruning threshold were the "sweet spots."
  • Focusing on "Hard" Tokens: Analysis showed that WISD naturally allocates more "supervision budget" to critical reasoning roles. Specifically, it places higher training weight on exploration (proposing tentative steps) and verification (checking intermediate results) tokens compared to their natural frequency in the data.
  • Implicit Error Correction: By contrasting with a version of itself that is prone to "forgetting" subtle logic or numerical precision, the model learns to double-down on the exact steps where its reasoning is most robust but fragile.

Key Results

WISD achieved consistent, state-of-the-art improvements across the board, often outperforming methods that do use privileged information.

  • Average Gains: Quantization and pruning improved average accuracy by 4.0 and 5.5 percentage points, respectively.
  • Math Performance: On the challenging HMMT 2025 benchmark, pruning raised Qwen3.5-4B’s accuracy from 57.7% to 70.5%.
  • Coding Performance: On LiveCodeBench v6, the same model jumped from 39.1% to 56.7%.
  • Scalability: The method works for large-scale MoE models too; Qwen3.6-35B-A3B saw its LCB v6 accuracy rise from 61.8% to 69.5% via quantization.
Figure 1. Accuracy gains from self-distillation against a weakened model. By constructing a weakened copy of the original model and using the contrast between them as a training signal, WISD achieves consistent accuracy improvements across diverse reasoning models and benchmarks. Full results are reported in Table 2.

Limitations and Future Work

  • Base Capability Requirement: WISD is a self-improvement framework. It likely requires a base model that already possesses a baseline level of reasoning ability to generate viable trajectories.
  • Hyperparameter Tuning: The effectiveness of the method depends on finding the right level of weight degradation, which might vary for different model families.
  • Future Directions: The authors suggest exploring more sophisticated weakening operators beyond simple pruning/quantization and investigating how WISD could be combined with verifiable reward signals (like unit tests in coding) to further boost performance.

Summary updated

Abstract

Self-distillation offers a way to improve reasoning models when a suitable external teacher is unavailable. However, common approaches rely on privileged information to construct a stronger self-teacher, limiting their use when only unlabeled prompts are available. In this paper, we introduce Weakening-Induced On-Policy Self-Distillation (WISD), which instead constructs a weaker counterpart of the original model and uses the contrast between the two as a training signal without requiring privileged information. We explore two ways to construct the weaker model: low-bit quantization and magnitude pruning. We evaluate WISD across diverse architectures and scales on mathematical and coding reasoning tasks. Across these benchmarks, quantization and pruning improve average accuracy over the original models by 4.0 and 5.5 percentage points, respectively, achieving state-of-the-art performance even without privileged information. Notably, pruning raises Qwen3.5-4B's accuracy from 57.7% to 70.5% on HMMT 2025 and from 39.1% to 56.7% on LiveCodeBench v6. Quantization raises Qwen3.6-35B-A3B's accuracy from 68.5% to 74.6% and from 61.8% to 69.5% on the same benchmarks, respectively. These results establish WISD as a novel and effective self-distillation framework for large reasoning models.
BibTeX
@misc{wu2026weaken,
  author = {Yecheng Wu and Song Han and Han Cai},
  title = {{Weaken to Strengthen: Weakening-Induced On-Policy Self-Distillation for Large Reasoning Models}},
  year = {2026},
  eprint = {2610.38681},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38681},
}
Figure