Many of the most damaging factual errors made by large language models (LLMs) are not invented facts. They are swaps. When a model writes a clinical summary about a kidney donor and a recipient, it may attach the donor's age to the recipient. When it summarizes a match, it may give the home team's score to the away team. Every value in the output exists somewhere in the source, which is why these errors are fluent, plausible, and hard to spot. This proposal treats the model's internal sense of which entity it is currently talking about as a position in a continuous space, rather than as a discrete label. A swap then shows up as this position drifting from one entity's region toward another's over the few tokens before the wrong value is emitted.
The core bet borrows from a line of work on latent categorical embeddings: models that place discrete categories (emotions, protein conformations, crystal shapes) as regions in a smooth low-dimensional space, so that "between two categories" is a real, measurable place. We claim that entity-role binding inside an LLM behaves the same way. If that is true, a small learned map of binding states, anchored to whatever roles appear in the current input, should detect role-swap hallucinations at the token level and ahead of emission. It should do so better than sentence-level truth detectors and fixed-class token probes. The same map also gives a principled, gated correction: push the state back toward the right role's region only when drift is detected.
Core Claim
Current methods that read truthfulness from model internals treat it as a single yes/no property of a whole answer. Inference-Time Intervention (ITI) fits a linear "truthful direction" in selected attention heads and shifts activations along it. Non-Linear ITI replaces the linear probe with an MLP and averages over the last few tokens. The Truthfulness Separator Vector (TSV) learns a tiny vector that pulls truthful and hallucinated sentence representations apart on a hypersphere. TSV's authors state that their method works at the sentence level on short QA and cannot localize entity- or span-level errors. MEDIMP, which uses ChatGPT to verbalize clinical tables, names donor/recipient swapping as a concrete risk it did not address.
Our claim is narrower and testable. In grounded generation with several entities of the same type, the model's middle-layer state at each generated token encodes a graded binding to one of the input's entities. Role swaps are preceded by continuous drift in that binding, visible one or more tokens before the wrong value appears. A continuous, context-anchored embedding of binding states captures this drift. A discrete clustering or a fixed-class classifier over the same activations does not capture it as well, especially when the set of roles changes from input to input.
This differs from the closest prior work in three ways. It is multi-category and context-relative rather than binary: the "categories" are the roles present in this input, not a global true/false axis. It works per token and uses the trajectory over time, not a sentence summary. It is explicitly about intra-source confusion rather than fabrication. HCMA reorganizes LLM token embeddings along a learned manifold and reports lower perplexity, but it never evaluates factuality and has no notion of per-context roles. It is the nearest "manifold meets LLM" work, and we differ in both target and evaluation.
Research Program
The method has three parts, each small.
The first part is anchors. For each input we take the residual-stream activation at the tokens that introduce each role (for example "donor", "recipient", or a person's name at first mention) at a band of middle layers. Layers 4 to 10 are where TSV found its separator effective on LLaMA-3.1-8B, and ITI found truthful heads in early-to-middle layers, so we search around that band.
The second part is the binding embedding. A small encoder $f$ maps the activation $h_t$ at generated token $t$, and each anchor $a_r$, into a shared space of 8 to 32 dimensions. The binding posterior over the roles $r$ present in this input is a distance softmax:
$$p(r \mid h_t) = \frac{\exp(-\lVert f(h_t) - f(a_r)\rVert^2 / \tau)}{\sum_{r'} \exp(-\lVert f(h_t) - f(a_{r'})\rVert^2 / \tau)}$$
Here $\tau$ is a learned temperature. The decoder is thus a categorical likelihood over a continuous latent, in the spirit of Categorical Normalizing Flows and the Categorical Latent Gaussian Process. Both map discrete items into continuous coordinates so that non-linear category structure becomes geometry; we reuse that design choice rather than their full machinery. Because roles are defined by anchors rather than fixed output classes, the same map applies to inputs with two, three, or five entities and to role names never seen in training.
On top of the per-token latent $z_t = f(h_t)$ we place a temporal smoothness prior, a Gaussian random walk across tokens. This follows the sequential emotion trajectories of Kim et al. and the continuity maps of MALCOM, both of which replace discrete state hops with smooth paths. The smoothed trajectory yields two signals at each token: the posterior mass on the wrong role, and the drift velocity toward a non-current role.
Training uses automatically labeled data (described below) in which the correct binding of each value token is known. Following TSV's finding that a few dozen labels go a long way, we also test a low-label regime of 32 to 256 labeled generations, with pseudo-labels on unlabeled generations.
The third part is gated correction. When the drift score crosses a threshold before a value token, we move $h_t$ a bounded step along the latent path toward the anchor of the role the surrounding text is committed to, which is recoverable from the preceding generated role mention. The step is decoded back through a linear pseudo-inverse of $f$. Nothing changes at other tokens. This targets the steering-strength problem documented for ITI, where large shifts make the model evasive ("I have no comment"), and for REVIS, where excessive steering produced literal artifacts like "image.jpg". REVIS already shows that gating by a risk score helps; our contribution is where the gate comes from and which direction the step goes.
The analysis half of the program asks whether the geometry is real. We test whether swaps are preceded by monotone drift or by abrupt jumps. We also test whether a truly continuous latent helps, compared with a discrete-state model fit to the same activations. If binding states form isolated clusters with no intermediate occupancy, the direction-1 thesis fails for this setting, and we will say so.
Data
No benchmark in the research file measures intra-source role swaps at the token level, so we build one and then check transfer to existing benchmarks. The constructed set, which we call RoleSwap, has three parts.
The first part is table-to-text inputs with two or more same-typed entities that share attribute types. Examples are donor/recipient clinical fields modeled on the MEDIMP setting, buyer/seller transactions, and home/away match statistics. Because every value is traceable to a cell, gold labels are automatic.
The second part is free-text sources in which the same relations are stated in prose, with distractor sentences and pronoun chains. A string match against a table is no longer trivially available here.
The third part is a held-out split with role names and role counts unseen during training.
Generations come from LLaMA-3.1-8B, the model used by TSV, sampled at moderate temperature so that natural swaps occur. We report the natural swap rate as part of the results. For transfer we use HaluEval's summarization and dialogue splits (HaluEval). We re-annotate a sample of hallucinated summaries to identify which ones are entity or attribute swaps. We use TruthfulQA (Lin et al.) and TriviaQA only as a check that the detector does not claim to solve general falsehood, which it should not.
Baselines
The naive reference point is the base model with no detector. Its natural swap rate on RoleSwap, and its token-level entropy or maximum-probability score as a detector, set the floor. We report the swap rate itself as a finding, because no reported result was found on Lacuna for token-level role-swap rates.
The prompt-based reference is asking a model to check its own output for swapped attributes, with the source in the prompt. HaluEval reports ChatGPT self-detection accuracy of 58.53% on summarization, with open models near or below 50%. We run this with LLaMA-3.1-8B and with a larger checker under the same inputs and report token-span F1 and AUROC.
The strongest existing internal-state detector is TSV. Park et al. report 84.2% AUROC for hallucination detection on TruthfulQA with LLaMA-3.1-8B (32 labeled examples) and 79.8% AUROC when transferred to TriviaQA. We run TSV with its released recipe, on the same model and layers, and with the same labeled budget as our method. We score it both at sentence level, its native mode, and at token level by applying its separator score to each token's state. Beating TSV means higher sentence-level AUROC on RoleSwap at an equal label budget, together with token-level localization that TSV cannot provide by design.
The strongest token-level competitor is a per-token probe trained on the same activations with the same labels. We use two versions. One is an MLP classifier with fixed role classes, in the style of NL-ITI. The other is a prototype probe that measures cosine similarity to the context anchors with no learned embedding and no temporal prior. We also run the adaptive token-selection detector released as mala-lab/HaMI; no reported result was found on Lacuna for it on any benchmark in this file. All probes get the same layers, label budget, and hyperparameter search budget as our method.
For correction, the baseline is ITI with its own truthful direction fitted on RoleSwap. We sweep its strength $\alpha$ and report swap rate against informativeness, as ITI did.
Two ablations isolate the key ingredient. The discrete ablation replaces the continuous latent with k-means or an HMM over binding states, which is the one-hot view that the latent-categorical literature argues against. The no-trajectory ablation keeps the embedding but drops the temporal prior. Our method counts as a success only if it beats the prototype probe and both ablations on early detection, meaning AUROC at one and two tokens before the value token, and on the unseen-role split.
Why not just…?
Why not just train a per-token classifier on which entity each token refers to? This is the real threat, and it is why the prototype and MLP probes are central baselines rather than afterthoughts. A fixed-class classifier cannot handle inputs whose role set differs from training. The prototype probe can, but it has no notion of trajectory. Our claimed advantage lives in the unseen-role split and in lead time before emission. If the prototype probe matches our method there, the continuous-manifold story adds nothing for this problem. We would then report the simple probe as the result and drop the manifold claim.
Why not just check the output against the source with string matching or an entailment model? For clean tables, a post-hoc string match will catch most swaps, and we include it as an upper-reference on the table split. It fails in two places our method targets. First, free-text sources with pronouns and paraphrase break simple matching; Kartchner et al. show extraction from clinical abstracts remains error-prone. Second, a post-hoc checker cannot act before the wrong value becomes conditioning context, and once the wrong value is in the context it compounds downstream, as Holt et al. describe. The free-text split and the lead-time analysis settle this comparison.
Why not just retrieve better context or ask the model to be careful? Retrieval does not help when every needed fact is already in the prompt, which is the defining property of a swap. Milecki flags swapping even with the structured source in hand. Prompted self-checking is measured directly in our prompt-based baseline, and HaluEval's numbers suggest it will be weak. HaluEval also found that showing correct and hallucinated answers side by side made discrimination worse.
Why not just use ITI to fix it? ITI pushes every token along one global direction. Swaps are context-specific, so the right direction for correcting "donor" differs from the one for "recipient". The ITI strength sweep against our gated correction will show whether a global shift can cut swaps without the loss of informativeness ITI itself reports.
Decisive Checks
Three results decide the project. The first is whether, on RoleSwap, the embedding's drift score predicts a swap one to two tokens before the value is emitted with AUROC clearly above the prototype probe and the discrete ablation. The second is whether that advantage holds on unseen role names and role counts. The third is whether gated correction lowers the swap rate on the free-text split while keeping informativeness and perplexity within a small margin of the untouched model, a trade-off ITI does not achieve at matched swap reduction.
If binding activations show no intermediate occupancy, meaning swaps appear as abrupt jumps with nothing between clusters, the manifold premise is falsified for LLM role binding. That negative result is still worth reporting, because it would tell the latent-categorical and interpretability communities where continuous category models do not apply.
References
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
- Non-Linear Inference Time Intervention: Improving LLM Truthfulness
- Steer LLM Latents for Hallucination Detection
- MEDIMP: 3D Medical Images and clinical Prompts for renal transplant representation learning
- Hierarchical Contextual Manifold Alignment for Structuring Latent Representations in Large Language Models
- Categorical Normalizing Flows via Continuous Transformations
- Latent Gaussian Processes for Distribution Estimation of Multivariate Categorical Data
- Estimating Temporal Dynamics of Human Emotions
- Maximum-Likelihood Continuity Mapping (MALCOM): An Alternative to HMMs
- REVIS: Sparse Latent Steering to Mitigate Object Hallucination in Large Vision-Language Models
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
- TruthfulQA: Measuring How Models Mimic Human Falsehoods
- Zero-Shot Information Extraction for Clinical Meta-Analysis using Large Language Models
- Data-Driven Discovery of Dynamical Systems in Pharmacology using Large Language Models