Systems that read whole-slide pathology images, scans that can exceed 100,000 × 100,000 pixels, cannot afford to examine every region at full magnification. The most efficient designs copy what a pathologist does: glance at a low-resolution overview, then zoom into the few places that look suspicious. The overview, however, is made with the same operator used to produce images for human eyes, a smooth averaging downsample tuned to look clean. Averaging is exactly what destroys a small cluster of tumor cells or a faint membrane stain. If a lesion disappears from the thumbnail, the system never zooms into it, and nothing later in the pipeline can recover the miss.
We propose to replace that human-oriented downsampling with a cheap learned operator whose output, a "machine thumbnail," is trained to keep whatever the high-magnification diagnostic model would have found informative. The training signal comes from a high-resolution model that already works. We run it offline on training slides and distill its patch-importance ranking into the low-resolution screener, using only slide-level labels. The claim is narrow and testable. At a fixed budget of high-magnification patches, a task-distilled downsampler recovers more lesion area during screening than any classifier, however strong, applied to a conventional thumbnail. The gain should concentrate on small lesions.
Where This Sits
Three bodies of work frame the proposal. The first motivates it. Work on the gap between human and machine perception shows that pipelines tuned for human viewing throw away what networks need. Hardware-in-the-Loop End-to-End Optimization of Camera Image Processing Pipelines tuned a physical camera's image processor directly against detection loss. The settings that were best for the machine kept high-frequency edges and looked noisier to people, and they gave a reported 30% mAP gain on KITTI over expert tuning. Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss (SR4IR) computes the super-resolution loss inside the downstream task network's feature space. Both methods change how the image is formed, but both work on natural images at moderate size, with dense labels or pixel ground truth. We borrow the principle that image formation should serve the task. We apply it to the reverse operation, downsampling rather than upsampling, and under weak supervision.
The second body supplies the architecture we modify. HAG-MIL scores patches at low magnification and uses a quadtree to zoom into the top-$k$ only. PTree-Net screens a thumbnail with a focal module before building a multi-magnification tree. Both build their overview with standard downsampling and score it with frozen features. A region that scores low at the coarse level is discarded for good. We keep their coarse-to-fine structure and change only what the coarse level sees.
The third body supplies the mechanism. Content-Adaptive Downsampling in Convolutional Neural Networks showed that keeping resolution only where content is informative matches dense accuracy at 35–70% less compute. Its masks, however, come from dense supervision or edge filters, and it was tested on Cityscapes-scale images. We need a signal for what is informative that exists without pixel labels. The high-resolution multiple-instance model provides one. Multiple-instance learning (MIL) here means learning from a slide-level label over a "bag" of thousands of unlabeled patches. Its attention weights mark which patches drove the slide decision.
Core Claim
Let $D_\theta$ be a shallow strided convolutional network applied once to the 20× pixels. Its output, with a few channels, is the machine thumbnail at roughly 1.25×–2.5×. Let $s_\phi$ be a screening network that assigns each coarse region an importance score. Let $a^{T}$ be the attention distribution over the same regions from a teacher MIL model, such as CLAM or TransMIL trained at 20× with slide labels only, pooled to the coarse grid. We train $D_\theta$ and $s_\phi$ with
$$ \mathcal{L} = \mathcal{L}_{\text{slide}}\big(s_\phi(D_\theta(X))\big) + \lambda\, \mathrm{KL}\big(a^{T} \,\|\, \mathrm{softmax}(s_\phi(D_\theta(X)))\big), $$
where $X$ is the slide, $\mathcal{L}_{\text{slide}}$ is the bag-level classification loss from the screener alone, and $\lambda$ weights the distillation term.
The distillation term sidesteps the non-differentiable top-$k$ selection, so reinforcement learning is unnecessary. The teacher's cost is paid only at training time. At inference, the downsampler costs a few operations per pixel. The deep encoder runs only on the coarse grid and on the selected high-magnification patches.
The new element is the downsampler itself. The thumbnail is no longer a fixed, human-oriented image. It becomes a learned representation, shaped by a weakly supervised teacher, that can encode sub-resolution evidence: local variance, stain-density peaks, nuclear-scale texture. Area averaging erases this evidence.
Research Program
We first build the teacher and the evaluation harness. A 20× CLAM-style and a TransMIL-style teacher are trained with slide labels on frozen self-supervised patch features, following the SimCLR recipe from DSMIL. We then measure screening recall: the fraction of annotated tumor area that falls inside the selected high-magnification set, as a function of budget $k$. We measure it for conventional thumbnails scored by a strong screener. Camelyon16's exhaustive tumor annotations are used only for this evaluation, never for training. This quantifies how much is lost before any zooming happens, a number the existing hierarchical papers do not report.
Next we train $D_\theta$ with the objective above. We vary its receptive field and the number of output channels to find how much capacity the information-preserving operator needs. Its cost is reported against the cost of encoding all patches at 20×.
Third, we place the learned thumbnail in front of HAG-MIL's zoom-and-aggregate stage and measure the slide-level accuracy–compute frontier.
As a secondary question, we replace fixed $k$ with an adaptive stopping rule: zoom until the cumulative screener probability mass reaches a threshold. This addresses HAG-MIL's arbitrary budget. We treat it as an add-on, not the core claim.
Baselines
The naive reference point is HAG-MIL's own pipeline: a standard pyramid thumbnail, frozen low-magnification features, and fixed top-$k$ zooming. HAG-MIL reports AUC 0.946 and F1 0.874 on Camelyon16 (397 WSIs), 0.921 AUC on TCGA-NSCLC, and 0.982 AUC on TCGA-RCC. We rerun it from the official code on our splits, with the same patch features and the same $k$ sweep. Beating it means higher screening recall at equal $k$, and equal or better AUC at a smaller $k$.
The strongest accuracy competitor is full 20× correlated MIL with no screening. TransMIL reports 93.09% test AUC on Camelyon16, 96.03% on TCGA-NSCLC and 98.82% on TCGA-RCC. CLAM with 80% patch annotations reaches 0.949 AUC in the mixed-supervision study, but it uses dense labels we do not. TransMIL is the ceiling we distill from, not something we expect to exceed. Success means matching its AUC within noise while encoding a small fraction of 20× patches.
The most important control isolates the key ingredient. We keep the same distillation loss and the same screener capacity, but feed the screener a conventional area-averaged thumbnail and let only $s_\phi$ learn. If this control matches the learned downsampler, then the gain comes from attention distillation, not from changing image formation. That would falsify the core claim.
We also include a capacity-matched control. The extra parameters of $D_\theta$ are added to the screener instead, so that the comparison is not won by parameter count.
All methods share data splits, patch features, the tuning budget (the same number of hyperparameter trials), and the count of high-magnification patches as the compute axis. Results are averaged over five seeds.
Why not just…?
Why not just encode everything at 20× and skip screening? This works for accuracy: TransMIL's numbers above show that frozen-feature MIL over all patches is strong. The proposal does not compete there. It competes on cost and latency, which matter once encoders are fine-tuned or large, and where slides must be triaged quickly. The experiment settles the question by placing every method on the same curve of AUC against patches encoded. If hierarchical methods with standard thumbnails already lie on TransMIL's curve at small budgets, the problem we target is not real. The screening-recall measurement in step one tests exactly this.
Why not just use a better encoder or distill attention onto the ordinary thumbnail? This is the main threat. DSMIL showed that in-domain self-supervised features beat ImageNet ones, and attention distillation alone may fix most screening errors. The isolating ablation in the Baselines section answers it directly. We expect any advantage of the learned downsampler to appear specifically on Camelyon16 slides whose largest lesion is below the coarse pixel footprint. We stratify by lesion size from the annotations, because those lesions are the only ones that area averaging erases beyond repair. If there is no gap on that stratum, we drop the claim.
Why not just super-resolve the thumbnail with a task-driven super-resolution model? Super-resolution can only reconstruct detail that is statistically implied by the thumbnail. A ten-cell cluster averaged into one pixel leaves little to reconstruct. Perceptual super-resolution can also invent misleading texture: the file notes that bilinear interpolation beat perceptual video super-resolution on action recognition. Super-resolution also adds an upsampling network at exactly the stage meant to be cheap. We include an SR4IR-style baseline, trained with the teacher's feature loss on the thumbnail and matched in compute, to show this empirically rather than by argument.
Why not learn where to zoom with reinforcement learning? ViGoRL and ACTIVE-o3 learn zoom policies for vision-language models. A policy still acts on what the overview shows, so it inherits the overview's blind spots. Our change improves the input to any such policy and is complementary to it. The adaptive-stopping variant tests the budget question with a much cheaper mechanism than reinforcement learning.
Proposed Experiments
The primary benchmark is Camelyon16. Its 270/129 split and pixel annotations allow screening recall and localization (FROC, with DSMIL's reported 0.437 as a reference) to be measured alongside slide AUC. TCGA-NSCLC and TCGA-RCC test whether the downsampler helps when the diagnostic signal is diffuse architecture rather than sparse foci. We predict little gain there, which makes these datasets a useful negative control.
The decisive checks, in order of importance, are three. First, recall and AUC on the small-lesion stratum of Camelyon16 for the learned downsampler versus the distilled-screener-on-ordinary-thumbnail control. Second, the AUC-versus-patches-encoded frontier against HAG-MIL and full-resolution TransMIL. Third, a cross-scanner check: train on one Camelyon16 center and test on the other, to confirm that the learned operator does not key on scanner-specific color statistics. Stain variation is a known failure mode, and a learned filter could amplify it.
We will also visualize what the learned thumbnail channels encode, to judge whether they track nuclear density or texture or simply re-learn stain color.
References
- Hardware-in-the-Loop End-to-End Optimization of Camera Image Processing Pipelines
- Beyond Image Super-Resolution for Image Recognition with Task-Driven Perceptual Loss
- Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification
- Diagnose Like A Pathologist: Weakly-Supervised Pathologist-Tree Network for Slide-Level Immunohistochemical Scoring
- Content-Adaptive Downsampling in Convolutional Neural Networks
- Dual-Stream Multiple Instance Learning Network for Whole Slide Image Classification With Self-Supervised Contrastive Learning
- TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification
- Attention-based Multiple Instance Learning with Mixed Supervision on the Camelyon16 Dataset
- Grounded Reinforcement Learning for Visual Reasoning
- ACTIVE-o3 : Empowering MLLMs with Active Perception via Pure Reinforcement Learning