When a slide-reading language model writes "tumor cells infiltrate the surrounding stroma with lymphocytic response," the sentence makes a claim about how three or more tissue regions sit together in space. It does not describe any single patch. Current whole-slide multimodal models encode thousands of image tiles as one long sequence and decode an answer in a single pass. Such sentences are exactly where they hallucinate. The usual cure is to have the model critique and rewrite its own answer, which roughly doubles response time. This note proposes a cheaper and more targeted cure. First, group the slide's tiles into multi-patch "neighborhoods": sets of tiles that are close in space and similar in appearance, the way a hypergraph joins several nodes with one edge. Then, after a first draft, re-examine only the neighborhoods that the draft's relational claims rest on, in one short verification pass, and rewrite only the sentences that fail.
The bet is that refinement compute should be spent where the draft's multi-region claims live, not spread across the whole slide or across every sentence. Most of what a pathology report says is either easy or non-relational. The costly, error-prone part is a small number of claims about tissue microenvironments, and those correspond to a small number of neighborhoods. If the bet holds, we get most of the hallucination reduction of full self-critique at a fraction of its latency. We also get a clean test of a disagreement in the literature: does self-correction help because the model looks at the image again, or because it writes its criticism out in words?
Core Claim
We claim that, for gigapixel vision-language tasks, a single scoped refinement pass will recover most of the accuracy gain of full critique-and-revise at well under half its added latency. The pass is driven by multi-patch neighborhoods implicated by the draft. We further claim that the gain will concentrate on relational claims (morphology, invasion, microenvironment) rather than on single-region facts. Finally, we claim that the unit of re-reading matters. A neighborhood returns tiles the model under-attended to, such as the stroma next to an attended tumor patch. Re-reading only the top-attended tiles cannot recover those.
Four lines of prior work set up this claim. Hyper-AdaC supplies the method for building neighborhoods. It clusters patches by both appearance and physical coordinates, compresses a slide to about 14% of its patches, and connects multi-patch communities with hyperedges. It uses them only for a single-pass survival estimate with no text. WSI-LLaVA and WSI-VQA motivate the work. They connect thousands of patch embeddings to a language decoder, through a long-sequence encoder or co-attention respectively. They do not model multi-region structure explicitly, and WSI-VQA itself flags hallucinated clinical text as a risk. Volcano both motivates and constrains the design. Its critique-revise-decide loop reduces hallucination on ordinary images, but it raises per-query time from 2.7s to 5.8s. Its qualitative finding is that verbal critique widens attention over image tokens. Meta-Reflection argues the opposite: verbal critique can be compressed into latent codebook units in a single forward pass. Its evidence comes only from text and code.
The closest structural precedent is Hyper-M2RAG. It restricts refinement to one-hop neighborhoods around cross-page anchor entities in parsed PDFs and cuts refinement tokens by 72–77%. Our proposal differs in three ways. Our neighborhoods are learned from continuous tissue geometry rather than from discrete layout blocks. They are selected per query by the draft's own claims, not by a frequency threshold. What gets refined is the answer, not the retrieval index.
Research Program
The system has three stages.
Offline neighborhood construction. Each slide is tiled, embedded with the same frozen patch encoder used by the base model, and grouped into neighborhoods with Hyper-AdaC-style clustering over appearance and coordinates. Each neighborhood is stored with its member tiles and its tiles' higher-magnification crops.
Drafting. The base slide-language model drafts an answer in one ordinary pass. While it does, we record cross-attention from each generated sentence to the patch tokens.
Scoped verification. A small claim tagger, trained on WSI-Bench report sentences, marks which draft sentences are relational. For each relational sentence $c$ and neighborhood $e$ we compute a relevance score
$$s(c,e)=\frac{1}{|e|^{\alpha}}\sum_{p\in e} a_{c,p},$$
where $a_{c,p}$ is the draft's attention from sentence $c$ to patch $p$, $|e|$ is the neighborhood's size, and $\alpha\in[0,1]$ controls how much large neighborhoods are penalized. We keep the top-$k$ neighborhoods per claim. The score is computed over whole neighborhoods, so a retrieved set includes tiles with near-zero draft attention that sit inside the same microenvironment. That inclusion is the specific mechanism we test.
The selected neighborhoods are re-encoded at higher magnification, which typically yields a few hundred tokens rather than thousands. A short verification pass then sees only these tokens, the claim, and the question. It emits a keep or revise decision and, where needed, a replacement sentence.
We will train the verifier in two variants, which directly tests the Volcano versus Meta-Reflection disagreement. The first variant writes a brief verbal critique before revising. The second skips the verbal critique and injects a learned latent reflection code, Meta-Reflection style, aligned to teacher critiques with optimal transport. Training data for both comes from perturbing correct WSI-Bench report sentences into plausible relational errors, for example swapping invasion status or the presence of an immune infiltrate. We also use the base model's own mistakes on held-out training slides.
We will analyze where time goes under three settings: the base model, full critique-revise, and scoped verification. Latency is measured on identical hardware and broken into slide encoding, draft decoding, re-encoding, and verification decoding. The target is an added latency of at most 1.3× the base model, against Volcano's 2.15×.
Baselines
Do-nothing reference. The do-nothing reference is the base slide-language model decoding once. We will use WSI-LLaVA's released checkpoint if available; otherwise we retrain it with its three-stage recipe on WSI-Bench. Its reported WSI-Bench numbers are an average score of 0.754, report BLEU-1 of 0.480, and WSI-Precision of 0.380. We first reproduce these within noise so that every later comparison shares the same backbone, patch features, and data split. On WSI-VQA (TCGA-BRCA), the reference points are W2T's tumor-subtyping F1 of 85.2 and TransMIL's 80.6 from WSI-VQA.
Strongest competitor on accuracy. This is full critique-revise-decide in the style of Volcano, applied to the same backbone. The model critiques its entire draft against the full slide token sequence, rewrites, and chooses between versions. It is the method most likely to win on accuracy because it spends the most compute on the whole answer. We fine-tune it on the same perturbation-plus-self-error data with the same number of training steps as our verifier. Volcano's reported numbers come from natural images (MMHal-Bench 2.64 versus 2.54 base, latency 5.8s versus 2.7s). No reported result was found on Lacuna for critique-revise on any whole-slide benchmark, so its pathology numbers will come from our own run. Beating it means one of two outcomes: matching its WSI-Precision and relational-claim accuracy within confidence intervals at less than half its added latency, or exceeding it at equal latency.
Hyper-M2RAG-style surgical baseline. We adapt Hyper-M2RAG to slides. Its refinement neighborhoods are fixed by a frequency-like heuristic: neighborhoods whose clusters recur across many queries. They are not selected by the draft's claims. This isolates whether claim-driven selection matters.
Key ablation: top-attended tiles. This is the ablation that isolates the proposal's main ingredient. It re-reads the top-attended individual tiles for each relational claim under exactly the same re-encoding token budget as our method, so the only difference is the unit of re-reading. If this ablation matches ours, the neighborhood structure contributes nothing.
Secondary ablations. We replace learned neighborhoods with fixed square windows of matching size, and we compare the verbal and latent verifier variants.
All arms share backbone weights, patch features, training data, tuning budget, and GPU type. Latency is reported as median and 95th percentile wall-clock time per query.
Why not just…?
Why not just run full self-critique and accept the latency? On accuracy, this is the real threat, and we expect it to be at least as accurate as our method. The question is cost. Volcano's loop more than doubles inference time even on single ordinary images. On gigapixel slides the critique pass must again attend over thousands of tokens. The comparison in this proposal settles the question with a quality-versus-latency curve. If full critique dominates our curve everywhere, scoped verification has no reason to exist and we will report that.
Why not just re-look at the patches the model already attended to? This is the simplest form of targeted refinement, and it is our key ablation at equal token budget. Our hypothesis is that relational hallucinations come mostly from regions the draft ignored, such as the stroma beside an attended tumor nest. Re-reading those regions requires a neighborhood unit. Prior hypergraph evidence supports multi-region units over pairwise ones: Hyper-AdaC beats pairwise graph baselines by at least 1.6% C-index on TCGA. That evidence comes from survival prediction, though, not from text. If top-attended re-reading matches us, we will drop the hypergraph component and keep the simpler method.
Why not just internalize reflection in one pass with a latent codebook and skip re-reading? Meta-Reflection cut first-token latency from 951ms to 246ms on text tasks. A codebook added to the base model would cost almost nothing. However, it never brings new visual evidence into the model, and Volcano's attention analysis suggests that fresh evidence is what drives correction on images. We include this arm with no re-encoding and the same training data. If it matches scoped verification, the bottleneck is reasoning rather than evidence and the proposal's premise is wrong.
Why not just feed more tiles at higher magnification in the first pass? We test this with a compute-matched single pass that spends our re-encoding budget up front, distributed by attention-guided sampling. If it equals our method, conditioning on the draft adds nothing.
Proposed Experiments
The primary benchmark is WSI-Bench. We report the average capability score, report-generation BLEU-1, ROUGE-L, WSI-Precision, and WSI-Relevance. We add a claim-level audit in which pathologists label about 300 held-out report sentences as relational or non-relational and as supported or unsupported by the slide.
Secondary evaluation uses the WSI-VQA dataset on TCGA-BRCA, reporting subtyping F1 and survival C-index. It also includes zero-shot transfer to CPTAC-NSCLC, for which no reported result was found on Lacuna. To check that the idea is not pathology-specific, we repeat the main comparison on VRSBench with an overhead-imagery backbone. The reference points there are LLaVA-1.5's captioning BLEU-1 of 48.1 and GeoChat's VQA accuracy of 76.0% after fine-tuning, from VRSBench. Neighborhoods there are built from object-region features and coordinates.
Three checks are decisive.
Where the gain falls. Gains should appear mostly on relational claims and on the morphological-analysis capability, with little change on non-relational facts. A uniform gain would point to generic extra compute rather than structure.
Unit of re-reading. At equal re-encoding budget, neighborhood re-reading should beat top-attended-tile re-reading on unsupported-relational-claim rate. Recovered errors should disproportionately involve tiles that had low draft attention.
Verbal versus latent. Comparing the two verifier variants decides whether verbal critique is necessary once the right evidence is back in view. If the latent variant matches the verbal one, the Volcano–Meta-Reflection tension resolves in favor of evidence over verbalization, and the fastest configuration becomes the recommended one.
References
- Hyper-AdaC: Adaptive clustering-based hypergraph representation of whole slide images for survival analysis
- WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Image
- WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering
- Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision
- Meta-Reflection: A Feedback-Free Reflection Learning Framework
- Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement
- VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding