Concise-9B: Removing the Slack from a Reasoning Model's Thinking

Alexander Boesgaard LoruparXiv 2026 · arXiv

TL;DR

Reasoning models often continue "thinking" long after they have internally arrived at the correct answer—a phenomenon the authors call "slack." By applying a simple self-training recipe that prioritizes the shortest correct reasoning traces, Concise-9B achieves the same accuracy as its base model (Qwen3.5-9B) while using 34% fewer tokens.

What Problem Does This Paper Address?

Modern "thinking" models (like DeepSeek-R1 or the Qwen-Think series) use test-time scaling: they generate long chains of thought to improve accuracy on complex tasks. However, this "thinking" is expensive. Previous research suggested that these models often "overthink," continuing to generate tokens even after the problem is solved.

The authors set out to quantify exactly how much of a reasoning trace is "slack" (tokens generated after the final answer is already fixed) and determine if a model can be trained to eliminate this waste without sacrificing its reasoning capabilities.

Methods

The researchers developed a recipe called concise self-training, which they applied to Qwen3.5-9B. The process involves:

  1. On-policy Rejection Sampling: The model generates multiple reasoning traces for a set of training problems.
  2. Shortest-Correct Selection: Instead of just keeping any correct answer, the fine-tuning process specifically selects the two shortest correct samples per problem.
  3. Answer-Synthesis Targets: To maintain robustness, 30% of the training data consists of "synthesis passes," where the model is shown multiple previous (potentially conflicting) attempts and asked to re-derive the final answer.
  4. Forced Readouts (Measurement): To measure slack, the authors used a "forced readout" technique. Every 2,048 tokens during a thinking trace, they forced the model to stop and provide a final answer. This allowed them to identify the "fixed point"—the moment after which every subsequent readout yielded the same final answer.

Datasets

The model was trained on a pool of 8,252 verifiable problems from MATH, GSM8K, MMLU, and GPQA.

Evaluation was performed on:

  • Development Suite: 498 problems from GPQA-Diamond, MATH, and MMLU.
  • Held-out Sets: A massive collection of 8,983 problems, including high-level competition math (AIME, HMMT), SuperGPQA, and MMLU-Pro.

Key Insights and Ideas

  • Slack is massive: In the base Qwen3.5-9B model, a staggering 54% to 64% of thinking tokens are generated after the answer has already stabilized.
  • Training beats inference-time rules: You might think you could just stop the model early at inference time (e.g., "stop if the last three internal answers agree"). However, the authors found this "hindsight" approach is brittle and causes a significant drop in accuracy (up to 9.2 points). Training the model to be concise is far more effective.
  • Efficiency through speed, not just cutting: Concise training doesn't just truncate the end of a trace; it actually makes the reasoning more efficient. The model reaches the correct answer earlier in the trace than the base model does.
  • The "Cheap Commit" Signal: The stabilization of forced readouts provides a reliable signal for "adaptive sampling." If a model's internal answer hasn't changed across the last two checkpoints, it's a safe bet to stop, saving compute without needing a second full sample.
Figure 1. Most thinking comes after the answer is fixed, and concise training removes mostly that part. (a) Mean thinking tokens per trace at a 32K budget, split at the point after which every forced answer equals the trace’s final answer; labels give the share after it (Table 4). (b) Share of traces whose answer is fixed by a given fraction of their thinking (hard and confirmation sets, 3,551 traces per model). A trace whose answer only settles at its end counts at 1.0.

Key Results

  • 34% Token Savings: At a 32K thinking budget, Concise-9B generates only 0.66x the tokens of the base model while maintaining equivalent accuracy (within ±1 point).
  • Maintained Accuracy: On the development suite, single-sample accuracy and majority voting remained equivalent to the base model despite the drastically reduced token count.
  • Slack Reduction: Concise training removed 86% of the targeted slack. In Concise-9B, the "after-fixed" portion of the trace dropped to 41-48%.
Figure 2. Development suite: accuracy against generated tokens per problem. Lines are majorities of 𝑘= 1, . . . , 4 traces (numbers mark 𝑘; 𝑘= 1 is the eight-draw single-sample mean); stars are escalation (2 →4). Values and paired intervals are in Table 1.

Limitations and Future Work

  • The Performance Ceiling: While Concise-9B is excellent for most tasks, it loses some "ceiling" on the absolute hardest competition math problems. When allowed to think up to 80K tokens, the base model eventually finds 25 more correct answers (out of 1,276) that the concise model misses. This suggests that for the most extreme edge cases, "verbose" thinking still has value.
  • Base Model Verbosity: The recipe's success depends on the base model. When replicated on Olmo-3-7B-Think (which is already quite terse), the token savings were much smaller (1–4%).
  • Measurement Resolution: The slack measurement is conservative because it relies on a 2,048-token grid; the actual answer might fix even earlier between those points.

Future work could explore more granular stopping signals or training methods that preserve the high-end reasoning "ceiling" while still eliminating slack on easier problems.

Summary updated

Abstract

Reasoning models answer better when they think longer, but a thinking trace keeps going well after its answer has formed. We measure this slack in Qwen3.5-9B and remove most of it with concise self-training: two rounds of on-policy rejection-sampling fine-tuning that keep the shortest correct samples, plus a small share of answer-synthesis targets, giving Concise-9B. On 8,983 held-out problems (competition and hard mathematics, SuperGPQA, a five-area confirmation set and MMLU-Pro), Concise-9B generates 0.66x the tokens of Qwen3.5-9B at a 32K thinking budget, with accuracy equivalent within +/-1 point (-0.20 [-0.77, +0.36] points). Forcing each trace on the hard and confirmation sets to answer every 2,048 thinking tokens, we find that 54-64% of Qwen3.5-9B's thinking tokens come after the point from which every forced answer equals the final one (41-48% for Concise-9B). Stopping there would keep every answer at 0.35-0.44x the base model's tokens, but finding that point takes hindsight: stopping when three consecutive forced answers agree costs the base model 2.4-9.2 points. Concise self-training removes slack without that cost: 86% [83, 90] of the thinking tokens it saves come after the answer has formed. It has a limit: with traces continued to 80K tokens, Concise-9B solves 25 fewer of 1,276 hard problems (-25 [-49, -1]), mostly in competition mathematics. The measurement also yields a cheap commit signal for adaptive sampling that matched two-sample escalation within +/-1 point at 0.67x its tokens in a pre-registered test. Results on Qwen3.5-4B, on which the recipe was developed, and a replication on Olmo-3-7B-Think suggest that the saving depends on how verbose the base model is.
BibTeX
@misc{lorup2026concise,
  author = {Alexander Boesgaard Lorup},
  title = {{Concise-9B: Removing the Slack from a Reasoning Model's Thinking}},
  year = {2026},
  eprint = {2611.02278},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.02278},
}
Figure