DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models

Yifeng Shen, J. Andrew Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jun Yan, Ning Wang, Kai Wang, Zhaoxiang Liu, Shiguo Lian2025 · DOI

Abstract

Large Reasoning Models (LRMs) utilize extended Chain-of-Thought (CoT) processes to solve complex problems, yet they frequently suffer from computational inefficiency due to "overthinking" simple tasks. Current methods to mitigate this—such as static length constraints—often degrade performance on high-difficulty problems. This paper introduces DAST (Difficulty-Adaptive Slow-Thinking), a framework that dynamically adjusts the reasoning token budget based on problem difficulty. By employing a Token Length Budget (TLB) and dual-preference optimization, DAST reduces average token usage by over 30% while simultaneously improving accuracy on challenging benchmarks like AIME 2024.

1. Introduction

The emergence of Large Reasoning Models (LRMs), such as DeepSeek-R1 and OpenAI’s o1, has marked a shift toward models that "think" before responding. While internal monologues improve performance on complex reasoning tasks, they introduce a significant efficiency flaw: the models often generate thousands of tokens for trivial problems. For instance, a model might produce a lengthy internal monologue for a basic arithmetic question, wasting compute and increasing latency.

Existing solutions typically involve "one-size-fits-all" compression, such as prompting the model to be concise. However, these static constraints create a trade-off where the model lacks the necessary "reasoning space" to solve PhD-level or competitive mathematics problems. DAST addresses this by creating a learnable mapping between problem difficulty and response length, allowing models to be brief when possible and verbose when necessary.

2. Methodology

DAST moves away from static length limits toward a dynamic, difficulty-aware framework consisting of four primary components:

2.1 Token Length Budget (TLB)

Instead of a fixed token limit, DAST calculates a dynamic budget for every query. The TLB is determined by a formula that balances the model's sampling accuracy ($p$) with the average length of correct responses ($L_r$). If a model consistently solves a problem correctly with few tokens, the TLB is restricted. Conversely, if the model struggles or requires more steps, the TLB expands toward the maximum generation limit. This ensures that difficulty is defined by the model's own performance rather than arbitrary labels.

2.2 Budget-Aware Reward Shaping

To incentivize the model to adhere to the TLB, the authors designed a reward function that applies penalties and rewards based on task difficulty. The model is penalized for exceeding the TLB on easy tasks but is rewarded for utilizing the full budget on difficult ones. This creates a "Goldilocks zone" that encourages efficient reasoning without sacrificing depth.

2.3 Dual-Preference Data Construction

The framework utilizes two types of contrastive pairs for preference optimization:

  • Dual-Correct Pairs (DCP): When two responses are both correct, the model is trained to prefer the shorter, more concise reasoning path.
  • Dual-InCorrect Pairs (DICP): When two responses are both incorrect, the model is trained to prefer the longer reasoning path. This is a critical innovation; it prevents the model from "giving up" too early on hard problems and encourages persistent engagement with difficult content.

2.4 SimPO Optimization

The model is fine-tuned using Simple Preference Optimization (SimPO). Compared to traditional RLHF methods, SimPO is more sensitive to length control, making it ideal for the fine-grained adjustments required by DAST.

3. Experimental Setup

The researchers evaluated DAST using DeepSeek-R1-Distill-Qwen (7B and 32B versions). Training was conducted on the MATH training set (approximately 10,000 pairs). The models were then tested on several high-difficulty benchmarks:

  • MATH-500: High-school competition problems across five difficulty levels.
  • AIME 2024: 30 extremely challenging problems from the American Invitational Mathematics Examination.
  • GPQA: PhD-level science questions (Physics, Chemistry, Biology) to test generalization.

4. Results and Discussion

4.1 Efficiency and Adaptivity

DAST achieved a reduction in token usage of over 30% on average across all benchmarks. Notably, the model demonstrated clear adaptivity: it reduced tokens by ~58% on Level 1 (easy) problems but only ~40% on Level 5 (hard) problems. This confirms that the model "knows" when to conserve resources and when to invest in deeper thinking.

4.2 Accuracy Gains

Surprisingly, reducing "fluff" on easy problems improved performance on hard ones. On the AIME 2024 benchmark, the 7B model's accuracy increased from 60.0% to 70.0%. This suggests that DAST helps the model focus on structured, high-quality reasoning. Furthermore, the model showed improved performance on the GPQA benchmark, despite being trained exclusively on math data, indicating that adaptive thinking is a generalizable reasoning skill.

5. Limitations and Future Work

While DAST shows promise in STEM domains, its efficacy in creative writing or general-domain tasks—where "correctness" is subjective—remains to be explored. Additionally, the framework currently relies on a manual grid search for the optimal truncation threshold ($\delta$) and utilizes off-policy data. Future research will focus on on-policy variants and expanding the domain scope to non-STEM fields.

6. Conclusion

DAST provides a robust framework for teaching LRMs to think only as much as necessary. By aligning reasoning length with problem difficulty, DAST solves the overthinking problem without compromising the model's ability to tackle complex challenges, offering a path toward more efficient and capable AI reasoning.

Summary updated

BibTeX
@misc{shen2025dast,
  author = {Yifeng Shen and J. Andrew Zhang and Jieyun Huang and Shuming Shi and Wenjing Zhang and Jun Yan and Ning Wang and Kai Wang and Zhaoxiang Liu and Shiguo Lian},
  title = {{DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models}},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-industry.160},
}
Figure