TL;DR
While Large Reasoning Models (LRMs) achieve stunning accuracy on static benchmarks, they are surprisingly fragile when interrupted or given new information mid-thought. This paper introduces a framework to evaluate LRMs in dynamic environments, revealing three distinct behavioral failures—reasoning leakage, panic, and self-doubt—that can cause performance to drop by up to 60%.
What Problem Does This Paper Address?
Standard evaluations of Large Reasoning Models (like DeepSeek-R1 or Qwen3) operate under a "frozen world" assumption: a model receives a fixed prompt, generates a long chain-of-thought (CoT) trace in isolation, and outputs a final answer.
However, real-world deployment is rarely static. A user might want to interrupt a long-running reasoning trace to get a quick partial answer, tell the model to "hurry up" under a sudden time constraint, or introduce new constraints mid-inference (e.g., updating a codebase or changing a math variable). Currently, the only way to handle these situations is to abort the generation and restart from scratch, which is highly inefficient. This paper investigates whether LRMs can gracefully handle real-time, mid-inference interruptions and context updates.
Methods
The authors formalize interruptible reasoning as a two-stage inference process. Instead of letting the model generate its full reasoning trace $r$, they halt generation at a specific relative point $X$ (e.g., 30% of the way through its typical token length) and inject interruption tokens or instructions $i$.
They evaluate two primary dynamic intervention scenarios:
- Time-constrained interruptions:
- Hard interrupts: Forcing an immediate answer by injecting structural tokens like
<end-thinking>or<force-answer>. - Soft interrupts: Providing a conversational directive to speed up (e.g., "Please answer faster").
- Update-driven interruptions: Injecting new, necessary task specifications or corrections mid-reasoning.
To help models adapt to update-driven interruptions, the authors also design a simple, training-free baseline called prompt guidance—a short postfix appended in the model's own voice confirming that the user's update is verified and correct.
Datasets
The evaluation suite spans two long-form reasoning domains across several standard benchmarks:
- Mathematics: GSM8K (a 500-example subsample), MATH500, and AIME-24/25.
- Programming: LiveCodeBench-v6.
For the update-driven scenarios, the authors used GPT-5 to programmatically split original benchmark problems into an altered initial problem ($p'$) and a mid-inference update ($u$), ensuring that the update was strictly necessary to reach the correct final answer. All augmented pairs were manually verified by the authors.
Key Insights and Ideas
The paper uncovers three novel behavioral pathologies when LRMs are forced to operate outside their comfort zone:
- Reasoning Leakage: When subjected to a hard interrupt early in their thinking process, models often fail to stop reasoning. Instead, they simply "leak" their remaining chain-of-thought into the final answer block—sometimes hiding it inside code comments or step-by-step text explanations. This results in answer lengths up to 10 times longer than normal, masking the true computational cost.
- Panic: When given a soft interrupt to speed up, models frequently experience an abrupt termination of logic. They prematurely close their thinking block almost immediately (using less than 1% of their remaining context budget) and output low-quality, incorrect answers.
- Self-Doubt: When given a valid problem update mid-inference, models often second-guess themselves or assume the user made a mistake. Instead of seamlessly integrating the new data, they waste tokens debating whether the update is correct, ultimately reverting to their original, now-incorrect line of reasoning.
Key Results
- Anytime Accuracy vs. Compute Hidden Costs: Under hard interrupts, LRMs generally show "anytime" behavior—meaning accuracy scales monotonically with the amount of reasoning allowed before the cut. However, because of reasoning leakage, early interruptions dramatically inflate the final answer length, meaning short-cutting the thinking block doesn't actually save as much compute as static metrics suggest.
- The Late-Update Penalty: For update-driven interventions, model performance degrades severely the later the update is introduced. When updates occur at the 90% mark of the reasoning trace, accuracy drops by up to 60% compared to a static baseline.
- Prompt Guidance to the Rescue: Introducing a short verification postfix ("prompt guidance") completely resolves the self-doubt pathology on easier tasks like GSM8K and MATH500, and significantly closes the performance gap on harder benchmarks like AIME and LiveCodeBench.
Limitations and Future Work
- Domain Scope: The study focuses strictly on math and coding because they generate long, reproducible reasoning traces. How these pathologies manifest in multi-turn dialogue, tool-use, or open-ended research agents remains an open question.
- Simplified Interruption Dynamics: Interruptions were modeled as single, clean events at pre-specified locations. Real-world user interactions are likely to be noisier, repeated, or potentially adversarial.
- API Constraints: The authors could only evaluate open-weight models (like Qwen and GPT-OSS) because proprietary commercial APIs do not currently expose or allow manipulation of intermediate reasoning states.
- Mitigation vs. Detection: The paper focuses heavily on identifying and categorizing these errors rather than fixing them fundamentally. The authors note that developing interruption-aware post-training and reinforcement learning objectives is a critical next step.
Summary updated