Vision-language models, which are chat models that take an image and write text about it, often describe objects that are not in the picture. The cheapest detectors read the model's own internal activations. A small classifier, called a probe, is trained on the hidden state of each sentence to say "grounded" or "hallucinated." Machine translation evaluation went through this same problem a few years earlier and learned something useful. A quality estimator that never sees a human reference can drift into judging whether the output sounds like a good translation instead of whether it matches the source. The field calls this source blindness. Our concern is that hallucination probes may share the defect. They may score how typical a sentence is for this kind of scene ("kitchens have refrigerators") rather than whether this particular image supports it. If so, they would be weakest exactly where hallucinations come from, namely the model's language priors.
We propose to test this directly and then fix it with the main tool of reference-free translation quality estimation: score the relationship between the source and the hypothesis, not the hypothesis alone. The test swaps each image for a similar scene that lacks the described objects and checks whether the probe's verdict changes. The fix runs the model a second time on the generated sentence with the image removed. This pass is teacher-forced, meaning the model reads the existing text rather than writing new text, so it costs one forward pass and no new generation. The probe is then trained on the difference between the image-conditioned and image-free hidden states, using free negatives made by pairing captions with mismatched images. The claim is specific: a probe that sees how much the image changed the model's internal state detects prior-driven hallucinations better than a probe that sees only the image-conditioned state, and it should stay robust to the count and attribute permutations that break text metrics.
What the Two Literatures Supply
Reference-free translation quality estimation supplies the method and the warning. COMET-QE (Are References Really Needed?) encodes source and hypothesis jointly and regresses human judgments. It matched its reference-based sibling on WMT21 zh-en (Pearson 0.720 vs 0.717), which shows that a well-designed source–hypothesis comparison can replace references. ALOPE adapts decoder language models to quality estimation with learned weights over several layers. It found that intermediate layers align source and target better than the final layer, and we reuse its layer-weighting scheme. The warning comes from the WMT22 robustness study (Exploring Robustness of Machine Translation Metrics). That study found 20 of 28 metrics below 0.4 correlation on challenge sets, with failures on number changes and permutations such as swapping two percentages across clauses. These are errors where the output stays fluent and plausible while contradicting the source. We borrow that challenge-set methodology for images. The closest bridging work, Multimodal Quality Estimation for Machine Translation, adds compressed ResNet features to translation quality estimation. It evaluates translations, not hallucination in open-ended generation, and it never asks whether the score depends on the image.
The hallucination literature provides the motivation and the competitors. Evaluating Object Hallucination attributes hallucination to co-occurrence priors and reports "yes" rates near 99% for some models. That means output probabilities are confidently wrong, which is why we do not rely on them alone. Reference-free Hallucination Detection for LVLMs shows that a linear probe on hidden states (which it calls SUQ, supervised uncertainty quantification) beats entropy and average-probability baselines by about 15% on sentence-level detection, and that it works best at the last layer. SHARP finds that the faithful/hallucinated split peaks in middle layers 10–15 and steers there. Neither paper checks whether its probe direction depends on the image, so our image-swap test applies to both. Heavier detectors take another route. UNIHD splits text into claims and checks each one with external tools under GPT-4V. CrossCheckGPT ranks models by agreement with other models. The M-HalDetect reward model (Detecting and Preventing Hallucinations) filters samples by best-of-64. These methods constrain our design: we aim to stay single-model and white-box, with one extra forward pass and no generation. Finally, Why LVLMs Are More Prone to Hallucinations in Longer Responses shows attention drifting from image tokens toward generated text in long outputs. That predicts source blindness should worsen late in a description, and we test this prediction.
Core Claim
Let $h_\ell(I, y)$ be the layer-$\ell$ hidden state of the model at the end of sentence $y$ when the image $I$ is present, and let $h_\ell(\varnothing, y)$ be the same state from a teacher-forced pass with the image tokens removed. The source-contrastive probe scores
$$ s(I, y) = \sigma\Big( w^\top \sum_{\ell} \alpha_\ell \big[\, h_\ell(I,y) \,;\, h_\ell(I,y) - h_\ell(\varnothing,y) \,\big] \Big). $$
Here $\alpha_\ell$ are learned layer weights in the ALOPE style, $[\cdot;\cdot]$ denotes concatenation, $w$ is a linear readout, and $\sigma$ is the logistic function. The difference term encodes how much the image moved the model's belief about this sentence. A sentence the model would have written about any kitchen moves the state very little, whatever the probe sees in $h_\ell(I,y)$ alone.
The probe is trained on two sources of labels. The first is human sentence and segment labels from IHAD and M-HalDetect. The second is free image-swap negatives. For a grounded COCO caption sentence, we retrieve a COCO image of the same scene category whose instance annotations lack the mentioned object, and we label that pairing hallucinated. These negatives are the multimodal analogue of the source perturbations in the WMT22 challenge sets, and they keep the image-free state constant within each grounded/swapped pair. That forces the probe to use the difference term.
Research Program
The work has three stages.
The first stage is diagnostic. We train the existing probes (SUQ last-layer and a SHARP-style layer-10 probe) on LLaVA-v1.5-7B as published. We then measure their swap sensitivity: the mean change in score, and the fraction of verdicts that flip, when a grounded sentence is re-scored against a swapped image. We report this separately for objects that commonly co-occur with the scene and objects that rarely do. The source-blindness hypothesis predicts that probes barely react to swaps involving common co-occurring objects. If they react strongly, the hypothesis is wrong, and the contribution narrows to the layer analysis and the challenge sets.
The second stage builds the source-contrastive probe described above and studies which layers it uses. The learned weights $\alpha_\ell$ give a direct, single-model answer to the disagreement between SUQ (last layer) and SHARP (middle layers). Our hypothesis is that the difference term concentrates in middle layers, where ALOPE also found source–hypothesis alignment, while the plain state is most separable late.
The third stage is a permutation challenge set modelled on the WMT22 study. We take sentences with counts and attributes, using COCO instance counts and the attribute and relation errors annotated in M-HalDetect. We create minimal edits such as swapping counts between two objects or swapping colors between two objects. These edits keep the set of words identical, so a probe that reads only lexical plausibility cannot separate them.
Baselines
The naive reference point is average token probability and predictive entropy, computed as in Reference-free Hallucination Detection and excluding punctuation tokens, which that paper found distort the score. The strongest single-model competitor is SUQ from the same paper. The file reports AUC-PR "in the 90s" on POPE and about a 15% margin over uncertainty metrics on sentence-level tasks. No exact reported IHAD or M-HalDetect value was found on Lacuna, so we rerun SUQ ourselves. It will use the same LLaVA-v1.5-7B checkpoint, the same training splits, and the same hyperparameter search budget, and it will get access to our image-swap negatives in one variant so that extra data is not confounded with the method. A SHARP-style middle-layer probe trained identically covers the other side of the layer dispute.
The most dangerous training-free competitor is the output-level analogue of our idea. It scores each sentence by $\log p(y \mid I) - \log p(y \mid \varnothing)$, a pointwise mutual information between image and text. This uses the same two passes as our probe and resembles the image-free contrast used in contrastive decoding. Self-consistency with NLI across five samples represents sampling-based detection. UNIHD with GPT-4V is the heavy tool-based ceiling; the file reports Macro-F1 81.63 on image-to-text MHaluBench, and we compare on that benchmark at matched granularity.
The key ablations remove the difference term (equivalent to SUQ with ALOPE layer weights), remove the swap negatives, and restrict the probe to the last layer only. Beating SUQ means a statistically reliable AUC-PR gain on IHAD and M-HalDetect, with a clearly larger gain on the swap and permutation challenge sets. Beating the probability-ratio baseline means the probe wins on those challenge sets, where any advantage of learned hidden-state features over the scalar log-probability ratio should be most visible.
Why not just…?
Why not just use the image vs. no-image log-probability ratio? This is the real threat, because it directly measures image dependence with no training. Two things count against it. First, output probabilities are badly calibrated in these models: the near-99% yes bias in Evaluating Object Hallucination and the punctuation distortion in Reference-free Hallucination Detection both show this. Second, a ratio summed over tokens dilutes a single wrong count across an otherwise grounded sentence. Neither argument is decisive in advance. The probability-ratio baseline runs on every benchmark, and if it matches the probe, we will report that the idea reduces to this cheaper score and that hidden-state features add nothing.
Why not just train SUQ on more data, including the swap negatives? Swap negatives might teach a single-state probe to be image-sensitive on their own. The ablation "SUQ plus swap negatives, no difference term" isolates exactly this. If it closes the gap, the contribution is the data recipe and the swap diagnostic rather than the architecture, and we would state it that way.
Why not just use CLIP similarity or an LLM judge? CLIP-style matching scores whole captions and is the global signal HALC already uses inside beam search (HALC). It has no obvious way to separate "two dogs, three cats" from "three dogs, two cats," and the permutation set tests that directly. LLM judges are reported to miss fine-grained hallucinations in Detecting and Preventing Hallucinations, and to invent errors themselves (Chen et al.). UNIHD stands in as the strong judge-plus-tools comparison.
Proposed Experiments
The primary model is LLaVA-v1.5-7B, so that results match the setup of SUQ and IHAD. InstructBLIP serves as the second model, since M-HalDetect descriptions were generated by it. Detection is evaluated by AUC-PR and AUROC on IHAD sentences, M-HalDetect segments, and POPE. Claim-level Macro-F1 is evaluated on MHaluBench image-to-text. Results are also reported on the two new challenge sets and broken down by sentence position, to test the long-response drift prediction.
Three outcomes decide the proposal. First, the diagnostic must show low swap sensitivity for existing probes on common co-occurring objects; otherwise the premise fails. Second, the source-contrastive probe must beat both SUQ-with-swap-negatives and the probability ratio on the permutation set; otherwise the hidden-state difference term is not needed. Third, the learned layer weights should place the difference term in middle layers.
A short downstream check uses the probe to rerank four sampled captions per image. It measures CHAIR sentence and instance rates together with object recall and caption length. The point is to confirm the reranker does not fall into the safe-response trap of vague, short captions. As a reference point, the M-HalDetect reward model reports a 15% hallucination reduction on LLaVA with best-of-64, and we make no claim to match that at N=4. Detection is the core claim. The cost of the method, one teacher-forced forward pass with the image removed, is reported in wall-clock time next to self-consistency sampling and UNIHD.
References
- Are References Really Needed? Unbabel-IST 2021 Submission for the Metrics Shared Task
- ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models
- Exploring Robustness of Machine Translation Metrics: A Study of Twenty-Two Automatic Metrics in the WMT22 Metric Task
- Multimodal Quality Estimation for Machine Translation
- Evaluating Object Hallucination in Large Vision-Language Models
- Reference-free Hallucination Detection for Large Vision-Language Models
- SHARP: Steering Hallucination in LVLMs via Representation Engineering
- Unified Hallucination Detection for Multimodal Large Language Models
- CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models
- Detecting and Preventing Hallucinations in Large Vision Language Models
- Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context
- HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
- MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark