Abstract
Large reasoning models (LRMs) have exhibited strong performance on complex reasoning tasks, with further gains achievable through increased computational budgets at inference. However, current test-time scaling methods predominantly rely on redundant sampling, ignoring the historical experience utilization, thereby limiting computational efficiency. To overcome this limitation, we propose Sticker-TTS, a novel test-time scaling framework that coordinates three collaborative LRMs to iteratively explore and refine solutions guided by historical attempts. At the core of our framework are distilled key conditions—termed stickers—which drive the extraction, refinement, and reuse of critical information across multiple rounds of reasoning. To further enhance the efficiency and performance of our framework, we introduce a two-stage optimization strategy that combines imitation learning with self-improvement, enabling progressive refinement. Extensive evaluations on three challenging mathematical reasoning benchmarks, including AIME-24, AIME-25, and Olym-MATH, demonstrate that Sticker-TTS consistently surpasses strong baselines, including self-consistency and advanced reinforcement learning approaches, under comparable inference budgets.
Introduction
Large Reasoning Models (LRMs) have demonstrated that increasing computational resources during inference—a process known as test-time scaling—can significantly improve performance on complex tasks. However, traditional scaling methods like "Self-Consistency" (Cons@N) often treat each reasoning attempt as an isolated event, leading to computational redundancy and a failure to learn from previous mistakes. Conversely, iterative methods that feed back entire reasoning chains often suffer from information overload, where the model becomes lost in its own verbose history.
Sticker-TTS addresses these challenges by introducing an abstraction layer called "stickers." These are concise, structured summaries of key conditions and strategies that allow the model to utilize historical experience without being overwhelmed by context length.
Methodology: The Sticker-Driven Framework
The Sticker-TTS framework operates through a Recursive Reasoning Loop involving three specialized modules:
- Sticker Extractor ($E$): Distills a reasoning trace into a structured "sticker" containing key conditions and strategies.
- Sticker Modifier ($M$): Acts as a debugger, refining the sticker to correct logical flaws or missing information.
- Sticker Utilizer ($U$): Generates a new, improved response based on the refined sticker and the original question.
Two-Stage Optimization
To optimize these modules, the framework employs a two-stage training strategy:
- Stage 1: Knowledge Distillation: Models are initialized using high-quality traces and stickers generated by powerful teacher models (e.g., DeepSeek-R1).
- Stage 2: Self-improvement Bootstrapping: The framework iteratively practices on datasets, specifically filtering for "Correction Significance." This ensures the model learns the critical skill of transitioning from an incorrect initial response to a correct one using sticker guidance.
Evaluation and Results
Sticker-TTS was evaluated on elite mathematical benchmarks, including AIME 2024, AIME 2025, and Olym-MATH. The results indicate that the sticker-driven approach provides a more efficient scaling path than standard majority voting.
Key findings include:
- Superior Accuracy: On AIME 2025, the 7B version of Sticker-TTS achieved a 12.42% relative improvement over standard self-consistency.
- Competitive Performance: The 32B model consistently outperformed reinforcement-learning-tuned models like Skywork-OR1 and Light-R1 across multiple benchmarks.
- Efficiency: Sticker-TTS achieved higher accuracy with 10 iterations than many baselines achieved with 64 independent samples, demonstrating the power of historical experience utilization.
Discussion and Insights
The success of Sticker-TTS highlights the importance of abstraction in reasoning. By forcing the model to condense its thoughts into "stickers," the framework prevents the "forgetting" issue common in long-context windows and prioritizes error recovery. While currently focused on mathematics, the principles of iterative refinement and diagnostic summaries offer a promising path for scaling reasoning capabilities in other complex domains.
Summary updated