How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?

Sohee Yang, Sang-Woo Lee, Nora Kassner, Daniela Gottesman, Sebastian Riedel, Mor Geva

Abstract

Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning, which is crucial in spotting mistakes and arriving at accurate solutions. A natural question that arises is how effectively models can perform such self-reevaluation. We tackle this question by investigating how well reasoning models identify and recover from four types of unhelpful thoughts: uninformative rambling thoughts, thoughts irrelevant to the question, thoughts misdirecting the question as a slightly different question, and thoughts that lead to incorrect answers. We show that models are effective at identifying most unhelpful thoughts but struggle to recover from the same thoughts when these are injected into their thinking process, causing significant performance drops. Models tend to naively continue the line of reasoning of the injected irrelevant thoughts, which showcases that their self-reevaluation abilities are far from a general "meta-cognitive" awareness. Moreover, we observe non/inverse-scaling trends, where larger models struggle more than smaller ones to recover from short irrelevant thoughts, even when instructed to reevaluate their reasoning. We demonstrate the implications of these findings with a jailbreak experiment using irrelevant thought injection, showing that the smallest models are the least distracted by harmful-response-triggering thoughts. Overall, our findings call for improvement in self-reevaluation of reasoning models to develop better reasoning and safer systems.

Introduction

Modern reasoning models, such as OpenAI’s o1 or DeepSeek-R1, generate extensive internal chains of thought (CoT) to solve complex tasks. This process is often assumed to facilitate "meta-cognition"—the capacity to reflect on reasoning steps, identify errors, and backtrack. However, it remains unclear whether this self-correction is a robust cognitive capability or a superficial pattern. This research investigates the gap between a model's ability to identify a flawed thought and its ability to recover from one when it occurs within its own reasoning stream.

Methodology

The study evaluates several model families, primarily DeepSeek-R1-Distill (7B to 70B parameters), alongside s1.1 and EXAONE Deep, across five rigorous benchmarks (AIME 24, MATH-500, ARC, GPQA, and HumanEval).

The researchers defined four categories of "unhelpful thoughts":

  1. Uninformative: Filler or rambling content.
  2. Irrelevant: Solving a completely different problem.
  3. Misdirecting: Solving a slightly altered version of the original problem.
  4. Incorrect: Containing logical or mathematical errors.

Two primary tasks were designed to test these models:

  • Identification Task: Models classify whether a given thought is helpful or unhelpful for a specific question.
  • Recovery Task (Thought Injection): The model's thinking block is "pre-filled" with an unhelpful thought, and the model must continue reasoning to reach a final answer.
Overview of the experimental setup for identifying (left) and recovering from (right) unhelpful thoughts.

Key Findings

The Identification-Recovery Gap

While models—especially larger ones—are nearly perfect at identifying unhelpful thoughts in isolation, they fail significantly when those thoughts are part of their own internal monologue. Instead of self-correcting, models often naively follow the flawed logic of the injected thought to an incorrect conclusion.

The Inverse Scaling Phenomenon

A surprising discovery is the "inverse scaling" trend regarding short irrelevant thoughts. Larger models (e.g., 70B) were found to be significantly worse at recovering from brief distractions than smaller models (7B). The larger models appear to "commit" more strongly to the injected tangent, whereas smaller models are more likely to ignore the distraction and return to the original prompt.

Performance across different unhelpful thought lengths, illustrating the inverse scaling trend where larger models struggle more with short distractions.

Shallow Meta-cognition

Even when models trigger an "aha moment" (realizing a mistake), they often only correct the reasoning locally within the wrong trajectory. They frequently fail to recognize that the entire premise of their current thought is irrelevant to the user's original intent. Interpretability analysis suggests that successful recovery is tied to maintaining strong attention on the original user query; larger models often lose this "anchor" once a distraction is introduced.

Results and Implications

  • Performance Collapse: Injecting unhelpful thoughts led to performance drops of up to 92% in extreme cases.
  • Jailbreak Vulnerability: In "attack-in-thought" experiments, 70B models were more easily manipulated into providing harmful responses via injected thoughts, while 7B models remained more robust.
  • Limited Efficacy of Prompts: Explicitly prompting for re-evaluation (e.g., "But wait...") improved performance for logical errors but failed to restore models to their baseline performance.

Conclusion

The findings suggest that current reasoning models lack true meta-cognitive awareness. Their susceptibility to internal distractions poses risks for both accuracy and safety. Future development should focus on training models to constantly ground their reasoning in the original user intent, ensuring they can resist being "gaslit" by their own internal tangents.

Summary updated

Abstract

Recent reasoning models show the ability to reflect, backtrack, and self-validate their reasoning, which is crucial in spotting mistakes and arriving at accurate solutions.A natural question that arises is how effectively models can perform such self-reevaluation.We tackle this question by investigating how well reasoning models identify and recover from four types of unhelpful thoughts: uninformative rambling thoughts, thoughts irrelevant to the question, thoughts misdirecting the question as a slightly different question, and thoughts that lead to incorrect answers.We show that models are effective at identifying most unhelpful thoughts but struggle to recover from the same thoughts when these are injected into their thinking process, causing significant performance drops.Models tend to naively continue the line of reasoning of the injected irrelevant thoughts, which showcases that their self-reevaluation abilities are far from a general "meta-cognitive" awareness.Moreover, we observe non/inversescaling trends, where larger models struggle more than smaller ones to recover from short irrelevant thoughts, even when instructed to reevaluate their reasoning.We demonstrate the implications of these findings with a jailbreak experiment using irrelevant thought injection, showing that the smallest models are the least distracted by harmful-response-triggering thoughts.Overall, our findings call for improvement in self-reevaluation of reasoning models to develop better reasoning and safer systems. 1
BibTeX
@misc{yang2025how,
  author = {Sohee Yang and Sang-Woo Lee and Nora Kassner and Daniela Gottesman and Sebastian Riedel and Mor Geva},
  title = {{How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?}},
  year = {2025},
  eprint = {2506.10979},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.10979},
}
Figure