Finding Rationalized Premises from the Shape of a Reasoning Trace

Reasoning models that "think out loud" before answering can do worse when handed a misleading hint. They do not ignore the false premise. They build an argument around it. On the SciAux science benchmark, Qwen3-8B with thinking enabled falls from 96.22% accuracy on clean questions to 77.03% when two misleading snippets are added, and ends up roughly level with the same model with thinking disabled (74.87%) (Thinking in a Crowd). We propose to detect and remove such premises by reading the structure of the model's own reasoning. A first-pass trace is turned into a small graph. Its nodes are the given facts, the auxiliary snippets, and the reasoning steps. Its edges record which step draws on which fact or earlier step. A classic graph-mining tool, the k-core (the largest subgraph in which every node has at least k neighbours, i.e. the densest mutually supporting nucleus), is then used to separate premises that sit inside a tight web of support from premises that act as hubs, touched by many steps but disconnected from the question's own facts. The model then reasons a second time with only the core premises.

The bet comes from keyword extraction. There, A Graph Degeneracy-based Approach to Keyword Extraction showed that important words live in cohesive cores rather than at high-degree hubs, and that core-based selection beats PageRank-style ranking by more than 5.5 F1 on short and medium documents. Our hypothesis is that rationalization has a recognisable shape of the same kind. The false snippet becomes a hub that takes over the argument, and the facts stated in the question drop out of the dense part of the trace. If that is true, a cheap, unsupervised, model-agnostic signal can catch distraction after the fact. Verifiers like Deductive Verification of Chain-of-Thought Reasoning cannot do this, because they trust the model to tag its own premises correctly, and a distracted model tags the wrong ones.

Core Claim

The claim has two parts. First, when a model rationalizes a misleading premise, the premise and the question's given facts stop sharing a dense core. We call this "anchor displacement." It should be measurable from the trace's premise-use graph and should separate misleading from helpful snippets better than how similar each snippet is to the question. Second, pruning premises outside the shared core and re-thinking recovers accuracy under misleading context. It must do so without discarding the helpful context that a "drop everything" policy throws away.

The second part matters because the trivial fix is already strong. With no auxiliary context at all, Qwen3-8B is back at 96.22% on SciAux. The method is only worth having on questions where some context is genuinely needed, and the experiments are built around that subset.

Relation to Prior Work

Several papers connect graphs to reasoning, but none uses graph structure to locate a misleading premise in the model's own trace.

Three papers use graphs over reasoning but not centrality over the trace. Graph of Thoughts uses a graph of thoughts as a control flow for merging and refining, with no structural ranking of nodes. Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning builds entity graphs from rationales with information extraction and learns node weights with a trained graph network inside T5. We use no training and work at inference time on decoder-only models. Verifying Chain-of-Thought Reasoning via Its Computational Graph shows that topological features including centrality predict step errors (92.47 AUROC on synthetic arithmetic). That graph, however, is built from internal transcoder circuits that are expensive and white-box only. We ask whether the visible text graph carries a comparable signal at a tiny fraction of the cost.

Two papers apply centrality to inputs rather than to reasoning. A Systematic Exploration of Knowledge Graph Alignment with LLMs in RAG finds that degree and PageRank centrality act as "reasoning anchors" when an external knowledge graph is written into a prompt. Its graph is fixed in advance and it never uses degeneracy. Mapping Text to Multiplex Graph (RAGP) prunes input sentences by random-walk visit frequency. It scores 49.3 on LongBench at 4x compression, against 48.8 for LongLLMLingua at 3x. RAGP ranks the input before any reasoning, so it cannot see which snippet the model is actually being drawn by. A topically on-message false snippet will likely look central to it. RAGP is our strongest input-side competitor.

Two papers motivate and constrain the work. Thinking in a Crowd and Can LLMs Always Solve Easy Problems if They Can Solve Harder Ones? (ConsisEval, where even GPT-4 Turbo reaches only 92.2% consistency under distraction) motivate the problem. Deductive Verification supplies the key methodological lesson: isolating each step with only its premises raises verification accuracy from 52% to 69% overall and to 84% on GSM8K. We replace its self-declared premise tags with structurally inferred ones.

Research Program

Building the graph. The model answers once with thinking on. The trace is split into steps. Premise nodes are the sentences of the question plus each auxiliary snippet. An edge from step to premise is added when the step reuses the premise's content. For black-box models this is measured by embedding similarity above a calibrated threshold together with shared numbers and entities. For open models we add a second edge type: summed attention from the step's tokens to the premise's tokens, in the spirit of RAGP's attention layer. Step-to-step edges link consecutive steps and steps that reuse each other's quantities.

Scoring premises. We compute the k-core decomposition of this graph. Each snippet $s$ receives an anchoring score

$$A(s) = \frac{|\{q \in Q : \text{core}(q) \ge \text{core}(s)\}|}{|Q|},$$

where $Q$ is the set of question-fact nodes and $\text{core}(\cdot)$ is a node's core number. $A(s)$ is the fraction of given facts that sit at least as deep in the graph as the snippet. Its companion is the snippet's degree. Anchor displacement predicts that misleading snippets have high degree and low $A(s)$, while helpful snippets have high $A(s)$. The cutoff is picked with the density-inflection rule (dens) from the degeneracy keyword work, so the method has no tuned top-k parameter.

Intervention. We remove the snippets that are low-anchoring, high-degree hubs and re-run thinking on the pruned context. If nothing is flagged, the first answer stands. This bounds the extra cost to at most one more generation, and only on flagged items.

The program has three stages. It first tests the diagnostic claim on its own, then the intervention, then transfer to ConsisEval-style distractors.

Baselines

The reference points. The naive reference is single-pass thinking on the full context, which gives 77.03% for Qwen3-8B with two misleading snippets on SciAux. Its counterpart is the no-context policy, which reaches 96.22% on SciAux overall. Because that number means context-dropping trivially "wins" on the standard split, we build a context-dependent subset: SciAux items, plus ConsisEval items with injected supporting facts, on which no-context accuracy is low and the helpful snippet is required. Thinking-off is reported throughout as a third reference.

Input-side pruning. The strongest existing input-side method is RAGP's random-walk pruning (Mapping Text to Multiplex Graph). We run it on the same contexts at a compression ratio matched to the fraction of snippets our method drops, with the same answering model and the same single extra generation. A dense question-similarity reranker is included as the simpler version of the same idea.

Model self-filtering. The strongest prompting competitor asks the same model, with thinking on, to judge which snippets are reliable before answering. We match its token budget to our two-pass cost. No reported result was found on Lacuna for self-filtering or self-consistency on SciAux, so both will be measured fresh.

Self-consistency. We take a majority vote over samples with the total generated tokens equal to our method's.

Ablations. Three ablations isolate the key ingredient. The first replaces k-core anchoring with PageRank on the identical trace graph, which tests whether cohesion rather than prominence matters. The second drops step-to-step edges. The third swaps the attention edges for lexical ones.

Success criterion. Beating the field means two things on the misleading-plus-helpful condition with confidence intervals over seeds. Our method must exceed RAGP and self-filtering at matched compute. It must also beat no-context on the context-dependent subset while staying within noise of no-context on the standard split.

Why not just…?

Why not just drop all auxiliary context? On standard SciAux this works, and we say so plainly. It fails by construction whenever context carries needed information, which is the normal case in retrieval-augmented reasoning. The context-dependent subset is designed to show this failure. If our method cannot beat no-context there, the contribution collapses.

Why not just ask the model which snippets to trust? This is the real threat. Strong models may simply recognise false science claims. The evidence against relying on it is indirect. Deductive Verification reports that unguided model verifiers are "lazy" and accept bad steps, and the rationalization result shows thinking amplifies commitment to false premises rather than rejecting them. The matched-budget self-filter baseline settles this directly. If it matches us, we will report that the structural signal adds nothing beyond the model's own judgment for this model size.

Why not just rank the input before reasoning, as RAGP does? Input centrality measures how well a snippet fits the topic. A plausible false snippet fits the topic well. Our signal comes from how the model actually uses the snippet. The diagnostic experiment compares the two AUROCs head-on.

Why not just use PageRank on the trace? Hub-sensitivity is exactly the failure that degeneracy was designed to avoid in keyword extraction, and a rationalized snippet is predicted to be a hub. The PageRank ablation tests whether this carries over.

Decisive Checks

The first check gates everything else. On SciAux with Qwen3-8B and Hunyuan-7B-Instruct, using the SciAux release, we measure the AUROC of the anchoring score for separating misleading from helpful snippets. We compare it against question-similarity, RAGP visit frequency, and trace-graph PageRank.

There is a specific way this could fail. If the model commits fully to a false premise, the snippet may sit inside the core alongside the question facts rather than at a hub. That would show up as $A(s)$ failing to separate the two classes. In that case we stop and report the negative result, because it would mean rationalization is structurally indistinguishable from sound reasoning in the visible trace.

If the first check passes, the second is the end-to-end accuracy comparison above. Results are broken out by the number of misleading snippets (zero, one, two) and by the context-dependent versus standard split. We also report response length, since the original study notes that rationalization comes with severe verbosity.

The third check is transfer. We test on ConsisEval easy problems with injected redundant premises, measuring recovery of the consistency score. We also correlate per-step anchoring with step-level errors on GSM8K-style chains, which gives a black-box, text-only counterpart to the computational-graph verifier.