Abstract
Reasoning Language Models (RLMs) utilize test-time scaling to generate long internal chains of thought, a process that remains opaque to end-users. This research introduces a framework for real-time progress prediction within these models. By integrating linear probing, supervised fine-tuning (SFT), and reinforcement learning (RL), the authors demonstrate that a 1.5B parameter model can be trained to communicate its internal state via a progress bar. The resulting system achieves a 10% mean absolute error (MAE) for reasoning sequences up to 16,000 tokens, providing a significant step toward transparency in complex AI reasoning tasks.
Introduction
Modern Reasoning Language Models, such as DeepSeek-R1 and OpenAI’s o1, have shifted the paradigm of AI interaction by spending significant compute time on internal "chains of thought" before delivering a final response. While this "test-time scaling" improves accuracy, it creates a "black box" experience for users who cannot distinguish between a model that is nearly finished and one that has just begun a lengthy calculation.
This research addresses the lack of transparency by investigating whether RLMs possess an intrinsic awareness of their progress and if they can be trained to report it reliably to the user.
Methodology
The researchers employed a three-stage approach to develop and validate progress-aware models:
- Internal Probing: To determine if progress information is already present, linear classifiers were trained on the model's hidden states. These probes successfully predicted the current "bin" of progress (e.g., 20-30%), suggesting that models naturally encode their position within a task.
- Supervised Fine-Tuning (SFT): A 1.5B parameter model was fine-tuned on mathematical reasoning datasets augmented with progress markers (e.g.,
<progressbar>35%</progressbar>). To ensure the model relied on context rather than simple token counting, "Noisy Labels" (random jitter) and masking of previous markers were introduced during training. - Reinforcement Learning (RL): Using Group Relative Policy Optimization (GRPO), the model was further refined. A tripartite reward system balanced prediction accuracy, formatting consistency (regular intervals), and the preservation of mathematical correctness.
Key Results
The study yielded several significant findings regarding the feasibility of real-time monitoring:
- Predictive Accuracy: The optimized model achieved a 10% Mean Absolute Error (MAE) for sequences up to 16,000 tokens. As shown in Figure 3, the combination of SFT with noisy labels and RL provided the most stable performance across varying sequence lengths.
- Stability and Format: While SFT-only models occasionally produced invalid predictions (e.g., progress exceeding 100%), RL-tuned models remained highly stable and adhered strictly to the required formatting.
- The Backtracking Phenomenon: Interestingly, models occasionally decreased their progress estimates. This "backtracking" occurred when the model identified a mistake in its reasoning and effectively "restarted" its internal calculation, demonstrating a genuine reflection of its cognitive state.
Discussion and Limitations
While the results are promising, the research highlights a "progress-performance trade-off." There is a minor decline in mathematical accuracy on the most difficult problems, suggesting that monitoring progress consumes a portion of the model's reasoning capacity. Furthermore, accuracy declines significantly on out-of-distribution sequences exceeding 32,000 tokens, indicating that the model's sense of progress is tied to the lengths encountered during training.
Future work will explore whether these techniques scale to larger models (70B+) and if progress awareness can be generalized to non-mathematical domains like creative writing or open-ended coding, where "the end" is less structurally defined.
Conclusion
By enabling reasoning models to report their own progress, we can transform the "black box" of test-time scaling into a transparent, manageable process. This work provides a foundation for better user interfaces and improved oversight of long-form AI reasoning.
Summary updated