When a smart-glasses assistant is asked "how much sugar is in this?", it first has to work out what "this" is, and only then look up the answer. Retrieval-augmented generation (RAG) means fetching documents from an external index and having the model answer from them. On egocentric multimodal benchmarks, it currently fails in one of two ways. Either the system confidently answers about the wrong product, or it says "I don't know" to almost everything. The best hallucination-suppressing system on Meta's CRAG-MM benchmark cut hallucinations from 59.62% to 2.88%, but refused 82.69% of questions (Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG). The competition winner's refusal-trained models decline 60–90% of queries (DB3 Team's Solution For Meta KDD Cup' 25).
Our claim is that much of both failures traces back to one link that current pipelines never check against the picture: the step that maps the image to an entity name, which then becomes the search key. Existing verifiers check the final answer against retrieved text. That cannot catch a fluent, internally consistent answer about the wrong entity. It also cannot give the system grounds to answer when the identity is in fact correct. We propose to verify identity visually. A dedicated verifier compares the cropped query object with reference images of each candidate entity, and checks any retrieved attributes that can be seen in the image, such as label text, colour, or logo. We then calibrate when to answer and when to refuse separately for the "is this the right thing?" link and the "is this the right fact?" link. The target result is the same hallucination rate as the most cautious systems, with a several-fold larger share of questions answered.
Core Claim
Over-refusal in multimodal RAG is a symptom of an unlocalized uncertainty. The CRUISE pipeline decomposes an answer into sub-claims with Chain-of-Verification (a method that splits an answer into small checkable claims) and refuses if any sub-claim fails (CRUISE). Because it cannot tell which link failed, it treats every doubt as fatal. DB3 has the same problem in a different form. It trains refusal into the generator with supervised fine-tuning (SFT), Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO), so the model learns a blanket aversion to answering. Its authors also note that it cannot separate visually similar entity classes (DB3).
We predict three things:
- A large share of CRAG-MM hallucinations are identity errors, where the right fact is retrieved for the wrong entity.
- A visual same-or-different verifier trained on hard negatives separates correct from incorrect identities far better than embedding similarity.
- Gating answers on a calibrated identity score and a calibrated fact-support score answers many more questions at a fixed hallucination budget than whole-answer refusal does.
Research Program
Diagnosis. The first component is a failure analysis on CRAG-MM, which provides egocentric images, a mock knowledge graph of 68K entries, and 800K web pages (CRAG-MM). We borrow MIRAGE's idea of decomposed filtration, which separates perception failures from reasoning failures in closed-book visual reasoning (MIRAGE), and move it to the open-book setting. For each wrong or refused answer from a straightforward RAG agent and from a CRUISE-style pipeline, we label the earliest broken link. The four labels are:
- the entity was misidentified;
- the correct entity was identified but retrieval missed the fact;
- the fact was retrieved but misread;
- the fact was read correctly but the reasoning over it failed.
We obtain these labels by intervention rather than human judgment alone. We substitute the gold entity name, then the gold passage, and see which substitution flips the outcome.
This diagnosis is a go/no-go gate. If identity errors account for only a small fraction of hallucinations and refusals, the proposal's premise is wrong. In that case we would redirect the calibrated two-link gating toward whichever link dominates.
Identity verifier. The second component is the verifier itself. Following DB3, we crop the object of interest with Grounding-DINO. The verifier then receives the crop, together with reference images and a short entity card for each of the top candidate entities returned by the benchmark's image and knowledge-graph search. It outputs a probability that the two show the same entity.
Training negatives are mined from the knowledge graph as entities of the same class that look nearly identical: same brand with a different variant, same species group, adjacent product models. This is exactly the confusion DB3 reports. A second, attribute head extracts visually checkable attributes from the retrieved text and asks whether each appears in the image.
This replaces cosine similarity in CLIP or BGE embedding space, which is how CMEGS scores cross-modal grounding (Adaptive Termination in Agentic Multimodal RAG via Cross Modal Evidence Grounding). That paper's own limitation is that similarity captures aboutness, not support: a photo of the wrong cereal box is still very "about" cereal.
Calibrated selective answering. The third component decides whether to answer. We take the method from TRAC, which uses conformal prediction (a distribution-free way to choose thresholds with finite-sample error guarantees) to bound answer-correctness errors in text RAG (TRAC). We repurpose it for abstention rather than prediction sets.
Let $s_{\text{id}}$ be the identity-verifier score and $s_{\text{fact}}$ the fact-support score, where the latter is a quote-grounded judgment that the retrieved passage supports the answer. Thresholds $\tau = (\tau_{\text{id}}, \tau_{\text{fact}})$ are chosen on a calibration split:
$$ \tau^\ast = \arg\min_{\tau}\ \Pr[\text{abstain}\mid\tau] \quad \text{s.t.} \quad \widehat{\Pr}[\text{hallucinate}\mid\text{answer},\tau] + c_n \le \alpha, $$
Here $\alpha$ is the target hallucination rate among answered questions. The term $c_n$ is a finite-sample correction for a calibration set of size $n$, taken from the binomial upper confidence bound used in conformal risk control.
When the identity score fails but several candidates remain plausible, the system has a third option between answering and refusing. It can ask one disambiguating question in multi-turn mode, or answer conditionally ("if this is X, then …"). We score conditional answers strictly under the benchmark's own grader.
Baselines
Naive reference points. The do-nothing reference is the benchmark's straightforward RAG agent. The CRAG-MM paper reports about 32% truthfulness for straightforward RAG on single-turn tasks, and says that GPT-5 and Gemini-2.5-Flash reach roughly the same level (CRAG-MM). The CRUISE report gives a truthfulness score of −34.62 for its baseline RAG agent (CRUISE). These two numbers come from different evaluation settings and are not directly comparable. We will therefore rerun all systems on one fixed public split under the official truthfulness grader, which awards +1 for a correct answer, 0 for an abstention, and −1 for a hallucination.
Strongest existing method. This is CRUISE: routing, retrieval with median-absolute-deviation noise thresholds, a dual-pathway judge, and Chain-of-Verification refusal. It reports a truthfulness score of 11.54, a hallucination rate of 2.88%, and a missing rate of 82.69% on CRAG-MM. It is the strongest competitor because it already achieves the hallucination level we target; the only open question is utility.
Competition winner. We also compare against DB3's refusal-aligned pipeline. For its overall truthfulness score, no reported result was found on Lacuna; the file reports only its refusal ranges and component scores.
Fairness. All systems share the same vision-language backbone (Llama 3.2-Vision, as used in the competition), the same mock APIs and per-query call budget, the same Grounding-DINO crops, and the same calibration and tuning data.
What counts as beating CRUISE. We must answer at least twice as many questions at a hallucination rate of 3% or lower. Equivalently, the full risk–coverage curve must dominate CRUISE's single operating point and a threshold sweep of DB3.
Ablations. Four ablations isolate the key ingredient:
- replace the trained identity verifier with CLIP cosine similarity, as in CMEGS;
- train the verifier without hard negatives;
- drop $s_{\text{id}}$ and calibrate on $s_{\text{fact}}$ alone;
- collapse both scores into one whole-answer threshold.
The last ablation tests whether localizing uncertainty matters, or only calibrating it.
Why not just…?
Why not just calibrate a single confidence threshold on the existing pipeline? This is the most serious alternative. Much of CRUISE's 82.69% refusal rate may simply reflect a poorly set operating point rather than a missing signal. The whole-answer-threshold ablation tests this directly: we sweep a single self-consistency or verbalized-confidence score under the same conformal procedure and compare risk–coverage curves. If the curves coincide, the identity verifier adds nothing, and the contribution reduces to calibration. We would report it that way.
Why not just use a bigger or frontier vision-language model? CRAG-MM reports that GPT-5 and Gemini-2.5-Flash stay around 32% truthfulness. MIRAGE finds that scale does not fix spatial-reasoning hallucinations. Neither result suggests that scale alone resolves fine-grained entity confusion. We will nonetheless include one frontier model inside our pipeline as the identity verifier, to separate the effect of model capacity from the effect of hard-negative training.
Why not just train refusal into the generator, as DB3 does? DB3's own numbers show that this produces 60–90% refusal. Training refusal into the generator also offers no handle for choosing a hallucination budget after training. Our design keeps the generator fixed and moves the decision to a separate, calibrated gate, so the hallucination budget $\alpha$ can be changed without retraining.
Why not just check grounding with embedding similarity (CMEGS)? CMEGS was validated on 30 queries, and its authors acknowledge that similarity does not establish entailment. The CLIP ablation measures how much of the identity signal similarity misses on hard-negative pairs.
Proposed Experiments
The main experiment reports risk–coverage curves on CRAG-MM, single-turn and multi-turn, at $\alpha \in \{1\%, 3\%, 5\%, 10\%\}$. We report empirical hallucination rate, missing rate, and truthfulness score, and check whether realized hallucination stays below $\alpha$ on a held-out split.
Several further checks follow. We test robustness on the low-quality egocentric images where CRAG-MM reports recall falling from 58% to 51.9%: does the verifier refuse appropriately there, rather than over-trusting blurred crops? We test transfer by running the identity verifier and gating, unchanged, inside an OmniSearch-style agent on Dyn-VQA, where OmniSearch reports 50.03% F1-Recall (Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent). There we ask whether filtering misidentified sub-queries reduces the planner's "thinking trap" loops.
The decisive check comes from the diagnosis study, rerun after intervention. If our system's remaining hallucinations shift from identity errors to reading and reasoning errors, while its answer rate rises at a fixed $\alpha$, the claim holds. If identity errors were never the bottleneck, or the single-threshold baseline matches us, the claim fails.
References
- Multi-Stage Verification-Centric Framework for Mitigating Hallucination in Multi-Modal RAG
- DB3 Team's Solution For Meta KDD Cup' 25
- CRAG-MM: Multi-modal Multi-turn Comprehensive RAG Benchmark
- MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
- Adaptive Termination in Agentic Multimodal RAG via Cross Modal Evidence Grounding
- TRAC: Trustworthy Retrieval Augmented Chatbot
- Benchmarking Multimodal Retrieval Augmented Generation with Dynamic VQA Dataset and Self-adaptive Planning Agent