Reasoning models given a misleading hint in their prompt often do not ignore it. They write long, fluent chains of thought that treat the false hint as a premise and reason carefully from it to a wrong answer. Increasingly, the model that decides whether such a trace is correct is another LLM acting as a grader, called an "LLM judge." This proposal asks two questions. First, how often do judges accept these well-argued wrong traces, and how confident are they when they do? Second, can we tell from the judge's own internal state that it has just been fooled, even when its written explanation gives no sign of it?
We answer both on a test set built from freshly generated problems whose every step is checkable. This matters for a reason that is easy to miss. On a public benchmark, a judge may reject a wrong trace simply because it has memorized the right answer. That memorization makes judges look more reliable than they are on anything new. The contribution has three parts: a contamination-resistant measurement of judge failure on rationalized errors, a direct estimate of how much memorization props up judge reliability on static data, and a test of whether a small classifier trained on the judge's internal activations flags wrong verdicts better than the judge's own stated confidence.
Core Claim
We hypothesize that judges will over-accept wrong traces specifically when the error comes from the student rationalizing an injected false premise. Wrong traces with ordinary errors should not show the same effect. We also expect the effect to be hidden by high verbalized confidence. Finally, we hypothesize that the judge's residual stream (the vector of internal activations passed between layers) carries a signal of "this verdict is wrong." A linear probe trained on one family of generated problems should read that signal and still find it on unseen problem families. If this holds, it is a practical monitor for judge failure that does not depend on the judge admitting anything in words.
Why This Gap Is Real
Several papers establish the ingredients and motivate the proposal. Thinking in a Crowd introduces SciAux, a set of multi-hop science questions paired with helpful, irrelevant, or misleading context. It names the "rationalization trap": Qwen3-8B falls from 96.22% to 77.03% accuracy with two misleading snippets when thinking is on, while its responses grow longer. The paper only studies solvers. It never asks whether a grader would catch these traces.
Distractor Injection Attacks on Large Reasoning Models shows something harder for a grader to spot. Reasoning can be hijacked internally while the final output looks normal, which the authors call covert compliance. The rate reaches 75% for DeepSeek-R1.
On the judge side, two papers show that judges are swayed without saying so and are badly calibrated. The Silent Judge finds that labels about a response's source or date shift verdicts by up to +30% while judges acknowledge the cue 0% of the time. ObjexMT measures judges' self-reported confidence on jailbreak transcripts and finds a 47.7% error rate among Qwen3-235B answers given with confidence of at least 0.90.
The closest existing work is ObjexMT. It combines judging, noisy context, and confidence, but it differs from this proposal in three ways. Its domain is safety intent extraction, not checking reasoning correctness. It reads confidence only from the judge's words. Its transcripts are static.
Two further papers constrain and supply the design. Preference Leakage shows that judges reward students from their own model family: the Preference Leakage Score reaches 37.1% on Arena-Hard, even though judges cannot identify their students in blind tests. So every judge–student pairing here must be controlled for model family. Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations shows that models can report on some low-dimensional, semantically meaningful directions in their activations. That suggests probeable structure exists, but it also warns that such structure may be narrow. Self-Knowledge Guided Retrieval Augmentation shows that simply asking a model whether it knows something is the weakest method in its comparison. This is our prior against relying on the judge's words.
For the method, we borrow from DyVal, which generates reasoning problems from randomly built dependency graphs (directed acyclic graphs, or DAGs) and shows that models strong on static benchmarks can collapse to near zero on them. We use its generator to build problems where the correct value of every intermediate step is known. That gives us ground truth for grading the judge.
Research Program
Building the test set. We extend DyVal's graph generator in arithmetic and boolean logic, plus a word-problem wrapper, to produce problems at complexity tiers D2–D4. For each problem, we write one or two misleading snippets that assert a wrong value for a specific intermediate node, in the style of SciAux. Because the corrupted node is known, we can automatically sort every student trace into one of three groups. A trace is correct. Or it is wrong and rationalized, meaning its step values match the injected wrong value and propagate it downstream. Or it is wrong for another reason. We also record whether the final answer is right while an intermediate step is wrong.
Students are Qwen3-8B and Hunyuan-7B-Instruct with thinking on and off, the models used in SciAux, plus DeepSeek-R1. Alongside the generated items, we use the static SciAux release ([billhdzhao/SciAux]) as the contaminated counterpart. For each generated family, we also build look-alike versions whose surface form resembles the static items.
Judging. Judges see the problem and the student trace and give a verdict of correct or incorrect, a step to blame, and a stated confidence between 0 and 1. We compare two judge conditions. In one, the judge sees the misleading snippet, as judges usually see the full input. In the other, the snippet is removed. This separates two effects: the judge being fooled by the student's argument, and the judge being fooled by the same noise the student saw.
We use open-weight judges from three families so that we can extract activations, including Qwen3 and Llama-family models. We add GPT-4o and a Claude model for behavioral results only. We cross judge and student families so that the leakage effect can be estimated and subtracted.
Monitoring. We record the judge's residual-stream activations at the token where it emits its verdict. At each layer, we train a logistic-regression probe to predict whether the verdict disagrees with the ground truth. The probe is trained on one generated family, arithmetic, and tested on the held-out logic and word-problem families and on static SciAux. We also check whether the probe direction lines up with a "misleading context present" direction. That check tells us whether the probe detects the judge's error itself or merely the presence of the snippet.
Measuring contamination support. For each judge, we compute accuracy on static SciAux minus accuracy on the matched generated look-alikes. We report this difference, the "memorization gap," separately for rationalized and non-rationalized wrong traces. If the gap is large for rationalized errors, then published judge reliability on static data overstates reliability on new data. That result stands whether or not the probe works.
Baselines
The naive reference point is a judge that always marks a trace correct. The next reference is a plain single-pass judge, whose stated confidence is taken at face value. Every monitor must beat both on the area under the ROC curve for detecting wrong verdicts, and on how often the judge is wrong while reporting confidence of at least 0.90 ([email protected]). The strongest published calibration numbers for verbalized judge confidence come from ObjexMT. There, claude-sonnet-4 reaches an expected calibration error of 0.206 and a Brier score of 0.254, and Qwen3-235B's [email protected] is 47.7%. That setting is jailbreak intent extraction. We found no reported result on Lacuna for judge calibration on rationalized reasoning traces, so we will produce this number ourselves. The ObjexMT figures serve only as a scale reference.
The strongest behavioral competitor is self-consistency. We sample the judge eight times at temperature 0.7 and use the rate of agreement as its confidence. We also include a stricter version that pools votes across judges from three different families. Both get the same test items. Their inference budget is matched to the probe's training data plus one pass per item, reported in tokens. A second strong competitor is the probability the judge assigns to its verdict token, which costs nothing extra.
We also include a heuristic designed to be hard to beat: a logistic regression on trace length and on how much the trace's wording overlaps with the misleading snippet. It is tuned on the same training family as the probe. We chose this because the rationalization trap is known to make traces longer, so length alone might explain most of the signal.
Two ablations isolate what matters in the probe. The first trains the probe on activations from the judge reading the same trace with the snippet removed. The second is a probe trained to predict only whether the snippet is present. The first tests whether the probe needs the noise to be in context. The second tests whether it detects errors rather than noise.
A reference-guided judge, which is shown the correct final answer, serves as an upper-bound oracle. We report it separately for traces where the final answer is right but an intermediate step is wrong, because final-answer checking cannot catch those. Beating the baselines means two things. The probe must exceed self-consistency on area under the ROC curve on held-out problem families by a margin larger than the spread across random seeds. It must also halve [email protected] relative to verbalized confidence, within bins of matched trace length.
Why not just…?
Why not just ask the judge how confident it is? The existing evidence says this fails exactly where we need it. Judges hide the cues that moved them, with 0% acknowledgment in The Silent Judge. Stated confidence is badly miscalibrated under adversarial context in ObjexMT. Direct self-assessment is the weakest strategy in Self-Knowledge Guided Retrieval Augmentation. Still, it is the main baseline, and we measure it directly on our data.
Why not just give the judge the reference answer? Where a reference exists, this is the right tool, and we use it as the oracle. It does not solve the deployment setting, where open-ended traces have no reference. It also cannot catch a wrong step that still ends at the right answer. Our static-versus-generated comparison tests a further possibility: on static benchmarks, judges may already be getting a hidden reference from memory. That would make reference-free judging look better than it is.
Why not just flag long traces or traces that repeat the snippet? This is a real threat, because rationalization inflates length. If the length-and-overlap heuristic matches the probe within length-matched bins, the monitoring half of the proposal fails. The paper would then be a benchmark and memorization-gap study, and we would report it as that.
Why not just sample the judge many times, or use a panel of judges? Disagreement among samples only helps when errors are uncorrelated. A persuasive rationalization may fool every sample the same way. Preference Leakage suggests that judges from the same family share blind spots. The cross-family panel baseline tests this directly, at matched cost.
Why not just let a stronger reasoning judge think longer? Longer thinking amplified the rationalization trap in solvers (Thinking in a Crowd). Longer reasoning also increased susceptibility to distraction (Distractor Injection Attacks). We run each open judge with thinking on and off to see whether judges inherit this effect.
Decisive Checks
Three results settle the proposal. The first is whether the false-acceptance rate on rationalized wrong traces exceeds the rate on other wrong traces within length-matched bins. That is the core behavioral claim. The second is whether the memorization gap between static SciAux and generated look-alikes is positive and larger for rationalized errors. That is the contamination claim. The third is whether an arithmetic-trained probe beats self-consistency and the length heuristic on held-out logic and word-problem families while the "snippet present" probe does not. That is the monitoring claim.
If the probe transfers to static SciAux but not to held-out generated families, it has learned features tied to the dataset rather than a general error signal, and we will say so. Each result stands on its own, so the study is informative even if only the benchmark findings survive.
References
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- Distractor Injection Attacks on Large Reasoning Models: Characterization and Defense
- The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
- ObjexMT: Objective Extraction and Metacognitive Calibration for LLM-as-a-Judge under Multi-Turn Jailbreaks
- Preference Leakage: A Contamination Problem in LLM-as-a-judge
- Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations
- Self-Knowledge Guided Retrieval Augmentation for Large Language Models
- DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks