Reading-State Audits at Agent Handoffs

In systems that split a task across several language model "agents" (a planner, a checker, an executor, each a separately prompted copy of a model), one agent's confident mistake is often accepted as fact by the next. This is hallucination snowballing across handoffs. Current defenses are either another agent that critiques the message in text, which is slow and tends to be fooled by fluent errors, or a detector attached to the agent that wrote the mistake. We propose looking somewhere nobody has looked: the hidden activations of the agent that reads the message. The bet is that a receiving model often registers internally that an incoming premise conflicts with what it knows, even when its text output goes along with it. If that registration can be picked up and summarized as a jump between a small number of discrete internal states, a system can decide at each handoff, for a few milliseconds of compute, whether to accept a message or send it back.

Concretely, we group the receiver's mid-layer hidden states into a few dozen clusters, which we call abstract states. We then fit two small Markov chains, one over state-to-state transitions seen while reading correct messages and one over transitions seen while reading corrupted ones. A message is scored by how much better the corrupted chain explains its trajectory. This reuses the state-abstraction machinery that has worked for single-model hallucination detection. The new element is the object being audited: the reader of an inter-agent message, which that machinery has never been applied to. The score is also used as a gate that triggers regeneration, not only as a passive alarm.

Core Claim

The receiving agent's hidden-state trajectory while it reads a peer's message carries information about that message's correctness. This information is not recoverable from the message text alone and is only partly recoverable from the sender's own activations. A discrete-state summary of that trajectory yields a handoff-level detector that is sample-efficient and cheap. It also stops more snowballs per unit of compute than a text critic.

One design choice makes this claim testable cleanly. We inject known-false premises into messages the sender never generated. For those messages, sender-side detectors have nothing to read, so any detection signal must come from the reader. For natural errors, we measure what the reader adds on top of the writer.

Relation to Prior Work

Several papers motivate the approach by showing that internal representations separate truth from fabrication in single models. PoLLMgraph clusters hidden states with principal component analysis followed by Gaussian mixture models, fits Markov models over factual and hallucinated generations, and reports beating ITI and SelfCheck by over 20% AUC-ROC on TruthfulQA using fewer than 100 reference samples. LUNA builds discrete-time Markov chains the same way, reports 90% ROC AUC for code hallucination detection on HumanEval, and finds that simple chains beat two-level hidden Markov models. CLUE shows that clustering activation changes across a reasoning block separates correct from incorrect solutions better than a GPT-4o judge (60.4% vs 54.0% on WebInstruct). We reuse the PoLLMgraph/LUNA pipeline almost unchanged; its sample efficiency matters because handoff labels are expensive. All three papers, however, audit the generator's own pass, whereas we audit a different model reading someone else's text.

What Do Latent Agents Actually Represent? supplies the key constraint on layer choice. In a four-agent system on Qwen3-4B, a correctness probe peaks at layer 9 (AUC 0.833) and weakens by the final layer. ICR Probe similarly locates hallucination signals in middle layers. We therefore cluster middle-layer states, not final ones. Lu et al. also report 75–80% redundancy between agents' representations. That is a warning: the reader's state might merely re-encode the sender's, and our ablations test exactly this.

On the multi-agent side, the Markov chain debate framework routes claims through Trust, Skeptic and Leader agents, reaching 76.00% on HaluEval Dialogue. Its "Markov chain" is a prompt-routing protocol over text, not over internal states. MACM inserts a text Judge between a Thinker and an Executor, and Self-collaboration chains an Analyst, Coder and Tester. These give us both the pipelines to instrument and the text-critic baselines.

Noël et al. show that log-probabilities and sampling agreement sit near chance (0.51–0.62 AUC) on confidently fabricated tool calls while hidden-state probes reach 0.947. They also explicitly leave multi-step compounding unstudied, which is the gap we fill. InEx is the only work that couples an internal signal with multi-agent verification. It uses a single attention-entropy scalar on the generator in a vision setting, not reader-side state dynamics.

Research Program

We instrument three open-weight pipelines whose models are large enough to hold their roles. Self-collaboration reports role collapse below about 13B parameters, so we use Qwen2.5-14B as the primary model and Llama-3-8B as a smaller-scale stress test. The three pipelines are a MACM-style Thinker→Judge→Executor pipeline on MATH, a two-agent decompose-then-answer pipeline on HotpotQA, and an Analyst→Coder→Tester pipeline on HumanEval and MBPP.

At every handoff we record the receiver's hidden states over the message tokens at a small set of middle layers, chosen on a held-out split. We reduce these states with principal component analysis and cluster them into $K$ abstract states. For each message trajectory $s_{1:T}$ of abstract states, we compute the handoff score

$$S = \frac{1}{T}\sum_{t=2}^{T} \log \frac{P_{\text{bad}}(s_t \mid s_{t-1})}{P_{\text{good}}(s_t \mid s_{t-1})},$$

where $P_{\text{good}}$ and $P_{\text{bad}}$ are transition probabilities estimated from receiver trajectories on correct and corrupted messages, and $T$ is the number of message tokens.

Labels come from two sources. The first is controlled injection: we replace one condition or intermediate result in a correct message with an entity swap, an arithmetic slip, or a plausible wrong fact, matched for length and fluency. The second is naturally occurring errors, labeled by checking intermediate claims against gold solutions, supporting facts, or unit tests.

When $S$ exceeds a threshold, the gate returns the message to the sender for one regeneration. We then track snowball depth, meaning how many downstream handoffs a corrupted premise survives. A secondary analysis asks whether particular abstract states correspond to recognizable reading modes. Examples would be "premise conflicts with stored fact," "premise accepted," and "premise unverifiable." We check this by inspecting which messages land in each cluster.

Baselines

The naive reference point is the ungated pipeline, which accepts every message. We also include a random gate that regenerates the same fraction of messages as our gate, which separates the value of which messages are sent back from the value of regenerating at all.

The strongest text-level competitor is the pipeline's own text critic. That means MACM's Judge on MATH and a Markov chain debate verifier on HotpotQA messages, run on the same base model with the same regeneration policy. The debate verifier reports 76.00% on HaluEval Dialogue and 72.00% on Factool QA. MACM reports lifting GPT-4 Turbo on MATH Level 5 from 54.68% to 76.73%. We rerun both on our open models rather than comparing against those numbers.

The strongest internal competitors are sender-side detectors. We run PoLLMgraph on the sender's generation, which reports a gain of over 20% AUC-ROC relative to ITI and SelfCheck on TruthfulQA. We also run ICR Probe, which reports 0.76–0.84 AUROC across HaluEval, SQuAD, HotpotQA and TriviaQA. Both get the same labeled budget and the same layer-selection search. Output-confidence baselines (sender token entropy, sampling agreement) are included because Noël et al. predict they will be weak.

Two ablations isolate the key ingredients. The first applies the same discrete-state pipeline to an unrelated model reading the message out of role context. This tests whether the signal comes from the reader's task-conditioned state or merely from the message text. The second is a linear probe on the receiver's continuous mean-pooled state, which tests whether discretization and transition modeling add anything over continuous features.

We report handoff-level results for all methods on identical messages and labels. No reported result found on Lacuna for handoff-level hallucination detection in multi-agent pipelines, so these numbers will be new.

Beating the field means three things. On injected errors, the reader-state score must exceed the text critic's AUROC. On natural errors, combining reader and sender features must exceed sender-only PoLLMgraph by a margin outside the bootstrap confidence interval. And at a matched regeneration rate, gating must raise end-task accuracy more than the text critic while using far fewer generated tokens.

Why not just…?

Why not just put a text critic at every handoff? That is what MACM and debate systems already do, and it is the most natural competitor. The evidence against relying on it is that fluent errors fool text judges: GPT-4o scores 54.0% on WebInstruct verification against CLUE's 60.4% (CLUE). In addition, HaluEval finds ChatGPT at only 58.53% on summary hallucination recognition. Our injected errors are built to be fluent, so the comparison is fair and direct. If the critic matches us there, our claim fails on accuracy and survives only on cost. We report both.

Why not just run a hallucination detector on the sender? This is the real threat. If errors are mostly visible in the writer's own state, reading the receiver adds cost for nothing. The injected-premise condition settles part of this question by construction, because the sender never produced those errors. For natural errors, the reader-plus-sender versus sender-only comparison settles it. We also check the Lu et al. redundancy worry directly by regressing reader states on sender states. If the reader's residual carries no correctness information, we will report that the reader signal is redundant.

Why not just a linear probe on continuous hidden states, skipping clustering? A probe may well match the discrete score on detection AUROC. Lu et al.'s layer-9 probe is exactly such a detector. If it does, the central finding that the reader's state carries a correctness signal still holds, and we would recommend the probe for detection. We keep discrete states only if they earn their place. That would require better accuracy in the under-100-label regime, where PoLLMgraph's sample efficiency suggests an advantage. It would also be justified if tracking a premise across multiple handoffs as a path through shared abstract states predicts snowball depth better than per-handoff probe scores do.

Why not just retrieve evidence? Retrieval helps on factual QA: HaluEval reports ChatGPT QA recognition rising from 62.59% to 76.83% with knowledge. It cannot check an arithmetic slip passed from Thinker to Judge on MATH. We include a retrieval-augmented critic on HotpotQA only, where it is applicable.

Proposed Experiments and Decisive Checks

The first experiment is injected-error detection at single handoffs on MATH, HotpotQA and MBPP, reporting AUROC for all baselines. The decisive check here is whether the reader score beats both chance and the text critic when no sender signal exists. The second experiment is natural-error detection, with the reader-plus-sender versus sender-only comparison as the decisive test of added value. The third is end-to-end gating, which measures final accuracy (MATH accuracy, HotpotQA exact match, HumanEval and MBPP pass@1), snowball depth, and generated tokens at matched regeneration rates.

Transfer tests fit the Markov chains on one task and apply them to another, and move from 14B to 8B models. These tell us whether the abstract states are task-specific or reflect a general "reading a false premise" mode.

A sensitivity sweep over $K$ and the number of principal components addresses the brittleness documented for state abstraction. The idea is rejected if the reader-only ablation run out of role context performs as well as the in-role reader, since that would mean the signal lives in the text, not in the agent. It is also rejected if sender-only detection explains away the reader's contribution on natural errors.