A whole-slide pathology image can exceed 100,000 × 100,000 pixels. Current models cope by cutting it into tens of thousands of small tiles at high magnification and running a pretrained network over every tile. Most of that reading is wasted, because a low-magnification overview already shows what most of the slide contains. We propose a model that looks at the overview first and fetches a high-magnification tile only when it expects that tile to change the diagnosis relative to what the overview already implies. The high-magnification view is treated as a costly message, and the model asks for it only when the message would be informative beyond what it already knows.
The idea comes from multi-agent communication. In collaborative perception between vehicles, RDcomm showed that a sender should transmit only features that are both useful for the task and not already available to the receiver. Doing this cut communication volume by up to 108× on OPV2V. Pathology's multi-resolution models select tiles on the first criterion only. They zoom where the overview looks suspicious, whether or not zooming would tell them anything new. We argue the second criterion, redundancy, is the missing one. A large, obvious tumor region is suspicious but needs no zoom. A faint, ambiguous focus is exactly where the overview is unreliable.
Core Claim
We claim the following: at a fixed budget of high-magnification tiles read per slide, a gate trained to predict the value of information of zooming will recover full-resolution slide-level accuracy with fewer reads than a gate that zooms on attention or suspicion. Value of information here means the expected change in the slide prediction when the overview-based guess about a region is replaced by its true high-magnification features. We also claim the advantage concentrates on slides with small, sparse lesions. If suspicion-ranked top-$k$ selection matches it at equal budget, the claim is false.
What is new is the selection target and how it is estimated. RDcomm tests redundancy with a learned mutual-information estimator in a two-round handshake, and the file notes that this adds latency. We avoid that cost. A small predictor learns to guess each region's high-magnification embedding from the overview, and a gate learns to forecast how much the error in that guess would move the slide prediction. Both are trained offline, so at test time the gate is a single cheap pass over the overview.
Mechanism
Let $x_r$ be the low-magnification (5×) embedding of region $r$, and let $z_r$ be the pooled high-magnification (20×) embedding of the tiles beneath it. Both come from the same frozen pathology encoder. A predictor $g$ maps $x_r$ and its spatial neighbors to a guess $\hat z_r$. The slide classifier $f$ is an attention-based multiple-instance aggregator, meaning a model that pools a bag of tile embeddings into one slide prediction using learned weights. It runs on a mixed bag in which each region contributes either $z_r$, if read, or $\hat z_r$, if not.
On training slides every region can be read, because features can be extracted once and cached. That lets us compute the true value of reading region $r$:
$$v_r = \left| f(\ldots,\hat z_r,\ldots) - f(\ldots,z_r,\ldots) \right|,$$
which is the change in the slide logit when only region $r$ switches from guessed to true features. A gate head $q_r = h(x_r) \in [0,1]$, which sees only the overview, is trained to rank regions by $v_r$. Jointly, $f$ is trained on bags in which regions are read with probability $q_r$, using a straight-through Bernoulli relaxation, under the constraint that the expected fraction read equals a target budget $b$:
$$\mathcal{L} = \mathcal{L}_{\text{task}}(f) + \alpha\,\mathcal{L}_{\text{rank}}(h; v) + \lambda\left(\tfrac{1}{N}\textstyle\sum_r q_r - b\right).$$
Here $N$ is the number of regions. The multiplier $\lambda$ is raised by gradient ascent whenever the budget is exceeded. This turns the bandwidth trade-off, which IMAC leaves as a hand-tuned "bandwidth dial," into a stated budget in tiles read. The term $v_r$ is a plug-in, task-weighted stand-in for the conditional information that high magnification carries given the overview: if $g$ predicts $z_r$ perfectly, $v_r$ is zero no matter how suspicious the region looks.
Relation to Prior Work
Two pathology lines motivate the gap. HAG-MIL builds a resolution quadtree and sends the top-$k$ low-magnification patches by attention score to higher magnification, reaching CAMELYON16 AUC 0.946. Its own authors note sensitivity to $k$. We keep its coarse-to-fine reading but change what is ranked, from suspicion to expected correction, and we let the number of reads vary per slide under an average budget. MuViT fuses multi-scale crops in one transformer with shared world coordinates. It shows that scales are complementary, but it reads all scales and never decides which to skip.
A third line supplies the method and also defines the closest competitor. Task-specific fine-tuning via Variational Information Bottleneck learns a Bernoulli mask that keeps a sparse, label-sufficient subset of patches so the backbone can be fine-tuned, reaching CAMELYON16 AUC 0.969. Its mask works at a single scale and selects for relevance only. Its purpose is to make training fit in memory, not to save reads at test time, where it re-extracts the whole slide. We reuse its Bernoulli-gate machinery but condition the gate on the overview and select for non-redundancy.
From the communication side, RDcomm contributes the principle that a message should be both task-relevant and new to the receiver. IMAC contributes the evidence that choosing who talks, without constraining what they say, fails under tight bandwidth. We transfer the principle but drop RDcomm's neural mutual-information handshake. In our setting the full-information training condition is available offline, so a supervised value-of-information target is cheaper and easier to check.
One finding cuts against our claim. MAMMOTH shows that a single linear projection after the encoder underfits heterogeneous tissue, so a single predictor $g$ may predict stroma well and tumor poorly. If so, the gate degenerates into a tumor detector. We test this directly, and we include a mixture-of-low-rank-experts predictor as a variant.
Research Program
We first precompute frozen encoder features at 5× and 20× for every slide and cache them. We then train $g$, $h$ and $f$ jointly at budgets $b \in \{2, 5, 10, 25\}\%$ of tissue regions. The main analysis is the accuracy-versus-reads curve. A second analysis checks calibration: on held-out slides we compute true $v_r$ by reading everything and measure rank correlation with $q_r$. A third analysis stratifies by lesion size using the CAMELYON16 annotations, to test whether savings come from skipping large obvious tumors and spending reads on micrometastases. We also measure wall-clock time, including slide I/O, because tiles not read is the quantity a deployment cares about.
Baselines
The naive reference points bracket the problem. The low-magnification-only aggregator reads no high-magnification tiles and sets the floor. Full high-magnification reading with the same frozen encoder and aggregator sets the ceiling our budgeted model should approach. Random selection at each budget isolates the value of any learned gate.
The strongest existing method for the question we ask is HAG-MIL-style suspicion-ranked top-$k$ zooming. It is the published design for spending a limited number of high-magnification reads. HAG-MIL reports CAMELYON16 AUC 0.946 and F1 0.874, TCGA-NSCLC AUC 0.921 and TCGA-RCC AUC 0.982. We found no reported result on Lacuna for HAG-MIL's number of high-magnification tiles read per slide, so we re-implement its selection rule on our cached features. The rule ranks by low-magnification attention and sets $k$ to match each budget. It uses the same encoder, aggregator, data splits and hyperparameter search budget as our method. Beating it means higher AUC at equal reads on at least two of three cohorts, or equal AUC with at most half the reads.
The second competitor is the overview-conditioned version of the VIB Bernoulli mask from Task-specific fine-tuning via VIB, trained with the same budget constraint. Its 0.969 CAMELYON16 AUC uses a fine-tuned backbone, so it is not directly comparable to frozen-feature models. We report it only as context and compare against a frozen-backbone re-run.
The key ablation keeps everything identical but trains the gate on a relevance-only target: the change in the slide logit when region $r$ is dropped, rather than guessed. This removes the redundancy term. If this ablation matches the full model, our central claim fails.
Why not just…?
Why not just read every tile with a strong frozen foundation model and skip the selection? On accuracy this is our ceiling, so it cannot lose. The question is cost. Reading tens of thousands of tiles per slide dominates inference time, and graph and attention models on such bags already force heuristic pruning: TMEGL caps input at 300 patches per tissue class. If our curve reaches within 0.01 AUC of full reading at 10% of reads, selection is worth it. If it does not, this alternative wins and we will report that.
Why not just zoom on the top-$k$ most suspicious regions, as HAG-MIL does? This is the real threat. On slides with a single obvious lesion, suspicion and value of information largely coincide, so top-$k$ may tie with our method. The disagreement should appear in two places: slides with large tumors, where top-$k$ wastes reads confirming what the overview already shows, and slides with small foci that the overview misses. The lesion-size stratification and the relevance-only ablation settle the question. If neither shows a gap, we drop the redundancy claim.
Why not just use a mutual-information handshake as in RDcomm? The file notes that neural mutual-information estimation adds latency that real-time systems struggle with (RDcomm). More importantly, a handshake would require reading the high-magnification tile to compute the test, which defeats the purpose. Our gate decides from the overview alone. If a handshake variant were more accurate, the calibration analysis would show $q_r$ ranking poorly against true $v_r$.
Proposed Experiments
We evaluate on CAMELYON16 (270 training and 129 test slides; AUC, F1 and lesion-size strata), TCGA-NSCLC subtyping, and TCGA-RCC. The reference numbers are DTFD-MIL's 0.946 and 0.961 AUC on CAMELYON16 and NSCLC from DTFD-MIL, alongside the HAG-MIL figures above. We use five-fold cross-validation where the cohorts allow, and all methods share cached features.
The decisive check is a single plot per cohort of AUC against mean high-magnification reads per slide, with our model, top-$k$, the relevance-only ablation and random selection overlaid. Our method succeeds if it dominates top-$k$ at the 2–10% budgets on CAMELYON16 and on at least one TCGA cohort.
The secondary check is whether, on CAMELYON16 slides with tumors larger than the median, our model reads measurably fewer tiles than top-$k$ at equal accuracy. The tertiary check is the rank correlation between $q_r$ and true $v_r$ on held-out slides, split by the predictor variant: single linear versus mixture of experts. This tells us whether MAMMOTH's interference effect limits the predictor.
References
- RATE-DISTORTION OPTIMIZED PRAGMATIC COMMUNICATION FOR COLLABORATIVE PERCEPTION
- Learning Efficient Multi-agent Communication: An Information Bottleneck Approach
- Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification
- MuViT: Multi-Resolution Vision Transformers for Learning Across Scales in Microscopy
- Task-Specific Fine-Tuning via Variational Information Bottleneck for Weakly-Supervised Pathology Whole Slide Image Classification
- Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning
- Tumor Micro-Environment Interactions Guided Graph Learning for Survival Analysis of Human Cancers from Whole-Slide Pathological Images
- DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification