Search engines and chat assistants have long assumed one person typing one query. Real information seeking often happens in groups: a team discusses a screenshot, a chart or a disaster photo in a shared chat, then asks an assistant to go find out more. That chat is full of claims ("the chart shows Q3 went up", "there's smoke behind the building"), and some of those claims are wrong. This note proposes that group chat creates a specific failure that single-user systems never face. A teammate's false statement about a shared image gets folded into the rewritten search query as if it were a given fact. The system then retrieves documents that agree with it, and a vision-language model (a model that reads images and text together) repeats the claim even though the image contradicts it. We call this presupposition laundering: the claim enters as chat, becomes part of the query, and comes back out as a sourced answer.
The proposed fix is simple to describe. Before anything reaches the query rewriter or the answer generator, the chat is split into two streams. One stream holds what the team wants: the information need and its constraints. The other holds what team members assert: who said what, about which artifact. Assertions are held apart. Each one is checked on its own against the image it refers to, by asking the model a yes/no question with the chat hidden. Only assertions that pass, or assertions explicitly marked "unverified, claimed by user B", may condition retrieval and generation. The contribution is a benchmark that measures laundering, plus evidence that this quarantine step, and not generic noise filtering, is what removes it.
Core Claim
Existing work treats noise in retrieval-augmented generation (RAG, where a model answers using passages fetched from an external store) as irrelevant or misleading documents. Existing work treats multimodal hallucination as the model's own language priors overriding the image. In collaborative search there is a third source that is neither of these: attributed human assertions in the conversation. Our claim is that such assertions are more damaging than equally false retrieved snippets, for two reasons. First, query rewriting turns them into presuppositions. Second, the documents retrieved with those presuppositions look like independent confirmation. Our further claim is that separating intent from assertion and verifying assertions without the chat in view removes most of this damage. It should do so while keeping the task-relevant information that simply discarding the chat would lose.
Prior work motivates each half of this claim. CoSearchAgent embeds a model in Slack and rewrites the last 20 chat messages into a standalone web query with cited answers. It is text-only and never asks whether the chat's claims are true, which is exactly the path through which laundering would occur. Thinking in a Crowd shows that misleading auxiliary snippets drag Qwen3-8B far below its clean 96.22% SciAux accuracy, and that explicit reasoning rationalizes rather than rejects them. Its snippets are anonymous text, not attributed claims about an image the model can see. Evaluating Object Hallucination in Large Vision-Language Models shows that models already over-answer "yes" for objects that often co-occur with the scene. We expect a teammate's assertion to push on that same weakness. Finally, HM-RAG and Adaptive Termination in Agentic Multimodal RAG build multimodal RAG systems, but for a single user with no conversational context.
Research Program
The first component is a measurement. We define the assertion-induced flip rate for a model $M$ as
$$\Delta_M = P_M(\text{yes} \mid \text{image}, \text{assertion}, \text{object absent}) - P_M(\text{yes} \mid \text{image}, \text{object absent}).$$
Here the assertion is a chat message from a named teammate claiming the absent object or attribute is present. $\Delta_M$ is how much the teammate's claim raises false "yes" answers above the model's baseline yes-bias. We will measure $\Delta_M$ three ways. The first is in isolation, as POPE-style probing (polling-based object probing: yes/no questions about whether an object is in the image). The second is after a CoSearchAgent-style query rewrite. The third is at the end of a full retrieval pipeline. The key diagnostic is the leakage rate: the fraction of rewritten queries that contain the false assertion as a presupposition. If rewriters rarely leak, the laundering hypothesis is wrong and the project reduces to a study of in-context distraction.
The second component is the benchmark, which we call CoMM-Noise. It is built from three sources that already exist.
POPE images and adversarial negative objects become simulated three-to-five-person chats. In each chat, one teammate asserts an absent co-occurring object, and other teammates contribute intent constraints and off-topic banter.
ScienceQA questions with diagrams, and their SciAux misleading snippets (billhdzhao/SciAux), become chats in which the misleading snippet is spoken by a teammate about the diagram. Each question is also posed with the same snippet delivered as an anonymous retrieved document. Comparing the two conditions tests whether attribution and conversational placement matter beyond content.
CrisisMMD image-text posts, which are informal user-generated disaster content, become coordination chats. Here teammate claims about damage or hazards are sometimes true and sometimes false.
Every item is labeled with whether the chat carries information needed for the answer, such as a constraint or a correct observation. That label is what separates quarantine from simply discarding the chat.
The third component is the method, assertion quarantine. A small parsing step turns the chat into intent slots and a list of attributed assertions, each tied to the artifact it refers to. Each assertion is converted into one or a few yes/no probes. The same vision-language model answers each probe against the referenced image alone, with no chat and no retrieved text in view. Retrieved documents are checked in separate, parallel branches in the style of Superposition Prompting, so one document's framing cannot contaminate another's. The query rewriter and the answer generator see the intent slots, the verified assertions, and the rejected assertions labeled as contradicted.
The added cost is one short image-only forward pass per assertion. This is typically far fewer calls than the roughly ten per question reported for debate-based RAG (Removal of Hallucination on Hallucination).
Baselines
The naive reference point is full-context concatenation. The entire chat, the shared image and the top retrieved passages go into one prompt, with no rewriting and no filtering. It shows how bad the problem is. DuetRAG reports that this style of unfiltered context lowered LLaMA2-7B HotpotQA accuracy from 12.7% to 9.2%, so we expect this baseline to be weak.
The most directly relevant baseline is CoSearchAgent-style rewriting (CoSearchAgent). The last 20 messages are rewritten into a standalone query, and the answer is generated from the retrieved results plus the image. It is the only published collaborative-search agent in the file, and it is the pipeline in which we claim laundering occurs. No reported result found on Lacuna for CoSearchAgent on any multimodal benchmark. We will rerun it with the same backbone, the same retriever and the same rewrite prompt budget as our method.
The strongest text-side noise filter is distraction-aware answer selection (Improving Zero-shot Reader by Reducing Distractions). It adds an "answer unanswerable if unsure" instruction and reweights each document's answer by how likely that document is to generate the query. It reports a 64% average exact-match gain over naive zero-shot readers. We also run Superposition Prompting, which reports a 43% accuracy gain and a 93x compute reduction over naive RAG on NaturalQuestions-Open with MPT-7B. Both filters act on retrieved documents, not on conversational assertions. They test whether generic relevance filtering already solves our problem.
The strongest visual-side baseline is RITUAL (RITUAL), which also feeds randomly transformed copies of the image at inference. It reports 88.87% POPE accuracy on LLaVA-1.5 against a base of 84.13%. We apply it on top of the CoSearchAgent-style pipeline.
For the multimodal RAG comparison we use HM-RAG, which reports 58.55% average accuracy on CrisisMMD. We run it with the chat concatenated into its input.
All methods share one backbone (LLaVA-1.5 for the POPE-derived portion, and a stronger open multimodal model for ScienceQA and CrisisMMD), one retriever and index, and the same prompt-tuning budget on a held-out development split.
Beating the baselines means lowering $\Delta_M$ and raising accuracy on adversarial items relative to the best of these baselines, without losing accuracy on items where the chat carries needed information.
Two ablations isolate the key ingredient. In the first, assertions are extracted but passed to the generator unverified; this tests whether separation alone helps. In the second, assertions are verified with the chat still visible; this tests whether hiding the chat during verification is what matters.
Why not just…?
Why not just drop the chat and keep only a clean rewritten query? This is the real threat. On items where the chat contributes nothing, it may match quarantine. Two things argue against it. First, a rewriter that reads the chat can still leak presuppositions into the query, and our leakage-rate measurement tests this directly. Second, a rewriter that ignores the chat loses constraints and correct observations. The benchmark's "chat carries needed information" split is designed to expose that loss. If query rewriting with a "strip all claims" instruction ties quarantine on both splits, we will report that the simpler method suffices.
Why not just use a long-context model and give it everything? The evidence in the file points the other way. Reader accuracy falls as more documents are added (Cho et al.). Vision-language models shift attention from image to text as context grows (Why LVLMs Are More Prone to Hallucinations in Longer Responses). Our naive baseline measures this directly on CoMM-Noise.
Why not just use a reasoning model that thinks through the contradictions? Thinking in a Crowd found that reasoning rationalizes misleading snippets. Its Qwen3-8B accuracy with two misleading snippets was 77.03% with thinking on, down from a 96.22% clean baseline. The file also gives 74.87% for thinking off while describing thinking-on as worse, which is internally inconsistent, so we rely only on the large drop from clean. We will run thinking on and thinking off as arms on the ScienceQA-derived portion.
Why not just use visual anti-hallucination decoding such as RITUAL? RITUAL targets the model's own priors, not an explicit claim sitting in the prompt, and it roughly doubles visual compute. If RITUAL alone drives $\Delta_M$ to near zero, then laundering is just ordinary yes-bias and our distinction collapses. That outcome is a clean, informative falsification.
Proposed Experiments
The first decisive check is whether $\Delta_M$ is substantially positive on POPE-derived chats. It should exceed the effect of placing the same false claim in an anonymous retrieved passage. If it does not, attribution in conversation is not special.
The second check is whether leakage rate predicts final error. It asks whether rewrites that contain the false presupposition retrieve confirming documents and yield wrong answers at a higher rate than rewrites that do not.
The third check is the head-to-head comparison. Quarantine should reduce adversarial-split errors relative to CoSearchAgent-style rewriting, distraction-aware selection, Superposition Prompting and RITUAL. It should also match or exceed chat-dropping on the needed-information split.
We report POPE accuracy, F1 and yes-rate, ScienceQA and SciAux accuracy, CrisisMMD accuracy, leakage rate, and model calls per query.
We will also collect a small set of real team sessions around shared images as a sanity check that the simulated chats are not the only place laundering occurs.
References
- CoSearchAgent: A Lightweight Collaborative Search Agent with Large Language Models
- Thinking in a Crowd: How Auxiliary Information Shapes LLM Reasoning
- Evaluating Object Hallucination in Large Vision-Language Models
- HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation
- Adaptive Termination in Agentic Multimodal RAG via Cross Modal Evidence Grounding
- Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
- Removal of Hallucination on Hallucination: Debate-Augmented RAG
- DuetRAG: Collaborative Retrieval-Augmented Generation
- Improving Zero-shot Reader by Reducing Distractions from Irrelevant Documents in Open-Domain Question Answering
- RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs
- Why LVLMs Are More Prone to Hallucinations in Longer Responses: The Role of Context