A pathologist reading a gigapixel slide does not decide every movement of the microscope from scratch. She runs a small set of habits: glance at the whole tissue, drop into a suspicious corner, sweep its neighbors, back out if nothing is there. Current learned systems for searching huge images lack such habits. A reinforcement-learning sampler picks one high-resolution patch per decision. A language-model agent writes each navigation step out in words. Both pay one full policy decision for every movement, and both learn search separately for each dataset. This note proposes to discover the habits automatically. We would record many search paths over image pyramids, then compress them, in the way a file compressor finds repeated substrings. What comes out is a small vocabulary of reusable, feedback-driven zoom skills. An image pyramid here means the same image stored at several resolutions.
The bet has two parts. First, search paths written in relative moves ("zoom into the upper-left child," "step right," "zoom out") are highly repetitive across slides, so a compression objective will find structure that a fixed chunk length cannot. Second, because each skill watches the image while it runs and decides for itself when to stop, a policy that chooses among skills should make several times fewer decisions than a one-patch-at-a-time policy at the same patch budget and accuracy. It should also carry over to new tissue types from a handful of slides. This mirrors what variable-length action compression did for robot manipulation.
Core Claim
Searching a multiresolution image can be cast as a sequence of relative navigation actions. A two-part minimum description length objective applied to those sequences yields a library of closed-loop search skills. Minimum description length (MDL) here means choosing the library that minimizes the bits needed to store the library plus the bits needed to re-encode all trajectories with it. A high-level policy that selects these skills should match the strongest single-patch sampler in slide-level AUC at equal patch budgets. It should cut the number of policy decisions per slide substantially, lower the rate at which tumor regions are wrongly discarded at coarse levels, and transfer to new pathology tasks with few labeled slides better than either single-step policies or fixed-length action chunks.
The closest existing work falls short in specific ways. Sequential Attention-based Sampling for Histopathological Analysis (SASHA) trains a PPO policy that queries 10–20% of patches. Its compression is of the final slide representation, not of the policy's behavior, and every action is a single isolated patch. CPathAgent and PathAgent produce explicit navigation paths. Their paths, however, are written by language models, synthesized with Gemini-2.5-Pro in CPathAgent's case, and are never distilled into a reusable vocabulary. PathAgent reports that this step-by-step prompting is slow. Unified Multimodal Understanding via Byte-Pair Visual Encoding does bring compression into vision. It applies byte-pair merging to 2D image tokens, using a spatial-consistency priority, but it builds static image tokens offline rather than skills that an agent executes. None of these discovers multi-step search behavior by compression.
Research Program
Action space. We place the slide in a quadtree pyramid. At any moment the agent sits at one node and has seen the low-resolution features of that node and its ancestors. The primitive actions are relative and image-independent: zoom into one of the four children, move to one of the eight same-level neighbors, zoom out, mark the current region as evidence, or stop. Relative actions are what make paths compressible across slides. Absolute coordinates, as in SASHA, would almost never repeat.
Demonstration paths. We need many search paths to compress, and we will build them from two sources. The first is privileged "oracle" searches on CAMELYON16 training slides. These use the lesion annotations to descend toward tumor while also making realistic exploratory detours and backtracks, with noise injected so the paths are not trivially optimal. The second is rollouts of a primitive-level reinforcement-learning searcher trained in the same relative action space. Using two sources lets us check whether the discovered skills reflect the structure of the search problem or quirks of one path generator.
Skill discovery. The method comes from Learning Options via Compression (LOVE). LOVE extends variational temporal abstraction with discrete skill variables $z$ and boundary indicators $m$, trained with Gumbel-Softmax and dual gradient descent to minimize
$$ \mathcal{L} = L(\mathcal{Z}) + L(\tau \mid \mathcal{Z}), $$
where $L(\mathcal{Z})$ is the description length of the skill library and $L(\tau \mid \mathcal{Z})$ is the length of the trajectory dataset encoded with it. LOVE showed that likelihood alone cannot tell "every action is a skill" apart from "every episode is one skill," and that the MDL term fixes this. Its experiments, however, used uniform-resolution renders and no multiscale perception. In our version, each skill's low-level policy and its stopping rule are conditioned on the coarse and fine features at the current node, so a skill such as "drill down while tumor-likeness rises, otherwise back out" reacts to what it sees.
This is a deliberate departure from PRISE. PRISE applies byte-pair merging to quantized robot actions and replays the resulting macros open-loop, a weakness it acknowledges. In search, that weakness is fatal, because every zoom decision should depend on the last view. Because our actions are already discrete, we also avoid PRISE's vector-quantization stage and the codebook collapse that comes with it.
High-level search policy. A PPO policy chooses skills rather than primitives. It uses SASHA's frozen feature distiller and reward setup so that the only difference from SASHA is the action vocabulary. Slide classification uses the patches the policy collected.
Analysis. We will inspect the discovered library: its size, its skill lengths, and whether skills match recognizable habits such as focal drill-down, boundary sweep, or backtrack. We will also test whether libraries learned from the oracle searches and from the policy rollouts agree.
Baselines
The naive reference point is uniform random sampling of the same fraction of high-resolution patches, followed by the same attention-based classifier. Any method that cannot beat it is not searching.
The strongest existing method is SASHA, run from its public code (coglabiisc/sasha). It is the strongest competitor because it already reaches very high accuracy with sparse, sequential, reinforcement-learned sampling. It reports 0.979 AUC on CAMELYON16 using 20% of patches, and its 10% variant is reported as 8x faster at inference than ACMIL or DTFD. We will rerun it with our feature extractor, splits, and patch budgets, and give it the same PPO tuning budget as our high-level policy. Beating it means matching its AUC within seed variance at 10% and 20% budgets while using clearly fewer policy decisions per slide, or exceeding its AUC in the few-slide transfer setting.
We also include SASHA's policy re-expressed in our relative primitive action space. This separates the effect of the action space from the effect of skills.
The strongest non-learned-search reference is HAG-MIL, run from its released code (bearcleverproud/hag-mil). HAG-MIL forwards a fixed top-$k$ of patches down a quadtree and reports 0.946 AUC on CAMELYON16 and 0.921 on TCGA-NSCLC.
Three ablations isolate the key ingredient, compression-chosen closed-loop skills. The first replaces skills with fixed-length chunks of $k$ primitive actions, the analogue of ACT, sweeping $k$. The second replaces MDL segmentation with likelihood-only variational temporal abstraction. The third keeps the MDL skills but executes them open-loop, as PRISE does, and also includes a PRISE-style byte-pair macro library. A fourth control is a small hand-designed macro set (descend-to-leaf, sweep-neighbors, backtrack), which tests whether compression finds anything a designer would not write down.
Why not just…?
Why not just keep SASHA's one-patch-at-a-time policy? At high budgets it may well tie us on AUC, and we expect it to; this is a real threat to any accuracy-only claim. The case for skills rests on two other measures. One is decisions per slide, which drive latency. The other is reuse: PRISE's gains were largest in few-shot adaptation to new MetaWorld tasks, 66.8% versus 59.3% success for fixed chunking. Accordingly, the decisive comparisons are decision count at matched AUC and the transfer experiments below. If SASHA matches us on both, the claim fails.
Why not just use fixed-length chunks of moves? This is the simplest temporal abstraction, and on LIBERO-90 it was not far behind compression: PRISE reports 54.4% versus ACT's 46.6%. Search might be regular enough that fixed chunks suffice. The fixed-chunk ablation, swept over chunk length, settles this directly. If its best setting matches the MDL library on decisions, AUC, and transfer, we will report that compression is unnecessary for search.
Why not just hand-code quadtree descent, as HAG-MIL and GlaZing effectively do? Hand-designed coarse-to-fine rules prune permanently. The research file names unrecoverable false pruning of faint targets as the central failure of these methods. Learned skills can include backtracking. We will measure annotated-lesion recall at each pyramid level, and the hand-designed macro control shows whether learned skills buy anything beyond a sensible fixed rule.
Why not just let a language-model agent navigate? CPathAgent's paths are useful, but each step costs a large-model call. PathAgent needed 1.21 iterations per diagnosis even after its efficiency improvements, and it degraded once more than 10% of patches entered its context. Our policy is a small network over a compact skill set. We include no language-model baseline on the classification tasks, because the two kinds of system optimize different outputs.
Proposed Experiments
The primary testbed is CAMELYON16. Its small metastases and lesion annotations allow both slide-level AUC and region-level false-pruning measurement. We will report AUC at 10% and 20% patch budgets, high-level decisions per slide, wall-clock inference time, and per-level lesion recall, against the baselines above over five seeds. The decisive check is a decisions-versus-AUC curve. Our method must reach SASHA's accuracy with substantially fewer decisions, and the fixed-chunk and open-loop ablations must fall measurably short. If open-loop execution performs as well, our closed-loop argument is wrong.
The transfer experiment freezes the CAMELYON16 skill library and trains only the high-level policy on BRACS, CamelyonPlus, and TCGA-NSCLC, using 5%, 10%, and 25% of the training slides. We compare against SASHA and fixed chunks retrained under the same budgets. There is no reported result found on Lacuna for SASHA on BRACS, CamelyonPlus, or TCGA-NSCLC, so all transfer numbers will be run fresh. For TCGA-NSCLC, HAG-MIL's reported 0.921 AUC with full training data serves as a ceiling. Success means the frozen library beats retrained primitive policies in the low-data regime.
A secondary check asks whether the library reflects search structure rather than one data source. We train the library on oracle paths and on policy rollouts separately, compare skill usage statistics, and test whether the oracle-derived library works when given to a policy trained from rollouts.
As an exploratory stretch, the same relative action space applies to GigaGrounding, where targets average 0.12% of the image. We would test whether skills discovered on pathology slides speed up search there, with the language query added to the high-level policy. GlaZing's reported 63.9% [email protected] is the reference. We treat this cross-domain result as a probe of how general the skills are, not as part of the core claim.
References
- Sequential Attention-based Sampling for Histopathological Analysis
- CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
- PathAgent: Toward Interpretable Analysis of Whole-slide Pathology Images via Large Language Model-based Agentic Reasoning
- Unified Multimodal Understanding via Byte-Pair Visual Encoding
- Learning Options via Compression
- PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control
- Diagnose Like a Pathologist: Transformer-Enabled Hierarchical Attention-Guided Multiple Instance Learning for Whole Slide Image Classification