Spending the High-Magnification Budget by Tissue Unit, Not by Tile

Pathology slides are enormous, often more than 100,000 pixels on a side. Current models cope by cutting each slide into thousands of square tiles and learning from a single slide-level label. The fast versions look at the slide at low magnification first and zoom into only the few tiles that look most suspicious. That shortcut has a known failure mode. If a small lesion looks unremarkable from far away, it is never examined up close. Square tiles also slice through the biological structures a pathologist actually grades, such as prostate glands, kidney glomeruli, and lung alveoli.

This note proposes a different unit of computation. At low magnification, first find the slide's functional tissue units (FTUs), the smallest structures that perform a physiological job. Then spend the fixed high-magnification budget unit by unit. Every unit is guaranteed a minimum look, and extra looks go to suspicious units. Crops are centered on units rather than aligned to a grid. The idea borrows the logic of part-based robot manipulation: locate the functional part coarsely, then spend fine-grained effort only on that part. The claim is that at equal compute, unit-based allocation misses fewer small lesions and grades gland-architecture tasks better than tile-based zooming. It also produces few enough instances, hundreds rather than thousands, that the whole slide can be modeled relationally without the arbitrary patch caps current spatial graphs impose.

Core Claim

A decomposition into functional tissue units, visible at low power, is a better budget allocator for gigapixel inference than tile-level attention. The advantage should appear as a better accuracy-versus-high-power-pixels curve and a lower miss rate on small lesions. It is not expected as a new record on saturated full-compute benchmarks. The claim is falsifiable in three ways. It fails if purely geometric segments (superpixels) allocate the budget equally well. It also fails if simply attaching unit-membership features to ordinary grid tiles matches it, or if tile-based zooming with the same budget matches it.

Why This Intersection, and What Is Already Done

On the robotics side, the lesson is that intermediate part structure makes coarse perception usable for fine decisions. CoPa first segments objects into functional parts (for example a kettle handle) and only then plans precise motions on those parts. It reports 63% task success against 18% for a monolithic baseline, and 11% when the part reasoning is removed. GAMap scores multi-scale image grids against part descriptions to find occluded targets. It runs at roughly 2 frames per second, which warns us that part reasoning must itself be cheap. These papers supply the principle of reasoning coarsely over parts before committing fine compute. They say nothing about pathology.

On the pathology side, three lines come close. None of them makes units the budget allocator.

Jain et al. define FTUs and benchmark segmenters across five organs, with a mean Dice of 0.835 and above 0.90 for glomeruli and crypts. They stop at segmentation and never feed units into slide-level diagnosis. We reuse their segmenters as the unit detector.

HACT-Net and its journal version build cell-to-tissue graphs and reach 61.53% weighted F1 on BRACS. They do this only on pre-cropped regions, because full-slide cell graphs would have millions of nodes. TMEGL builds tumor, stroma, and lymphocyte graphs for survival prediction, but caps each class at 300 patches for memory reasons. These papers motivate the proposal: biological structure helps, and TMEGL's C-Index drops from 0.719 to 0.609 on TCGA-BRCA without its neighborhood embedding. They also show the scale wall we target.

HAG-MIL is the method we directly compete with. It zooms through a quadtree of square tiles into the top-$k$ low-power regions and reaches 0.946 AUC on Camelyon16. Its authors acknowledge that tissue pruned at low power is gone for good. HGPMIL adds morphology scores to patch features and lifts ABMIL's TCGA-BRCA AUC from 0.825 to 0.839. That result implies an important competitor: keep the tiles and just add structural features.

One cited result cuts against us. CLAM already reaches 0.991 AUC on TCGA-RCC using plain grid tiles at full resolution. We should not expect accuracy wins on such tasks at full compute. The contribution lives in the budget-limited regime and on architecture-graded tasks.

Research Program

We would build a system with three stages. The first is a low-power pass, at about 1.25× to 2.5× magnification, that runs an FTU segmenter to produce unit masks. The segmenter is trained on the HuBMAP organs; for organs with no FTU annotations it runs class-agnostic. The same pass computes a cheap suspicion score $s_u$ for each unit $u$ from low-power features. The second stage allocates a fixed budget of $B$ high-power crops across the unit set $U$:

$$n_u = m + \left\lfloor (B - m|U|)\,\frac{\exp(s_u/\tau)}{\sum_{v\in U}\exp(s_v/\tau)} \right\rfloor ,$$

Here $n_u$ is the number of 20× crops sampled inside unit $u$'s mask, centered on the unit rather than grid-aligned. The floor $m \ge 1$ guarantees every unit is examined at least once, and the temperature $\tau$ trades exploration against focus. When $|U|$ exceeds $B/m$, units are merged by spatial clustering. The coverage floor is the direct answer to HAG-MIL's permanent-discard failure. The third stage is an aggregator. Crops pool into one embedding per unit, and a light attention-and-graph layer runs over units, with edges between adjacent units. Because there are hundreds of units, the whole slide fits in memory with no per-class cap.

The analysis has two parts. First, we trace accuracy against $B$ from very small budgets up to full-slide reading. Second, we measure lesion coverage, the fraction of annotated lesion area that received at least one high-power crop, stratified by lesion size. This second measurement tests the mechanism directly rather than inferring it from accuracy.

Baselines

The naive reference point is uniform random sampling of $B$ high-power grid tiles from the tissue mask, followed by attention pooling. Any allocator that fails to beat random at equal budget is not allocating.

The full-compute upper reference is CLAM (Lu et al.) reading every 20× tile. It reports 0.953 AUC on Camelyon16, 0.956 on TCGA-NSCLC, and 0.991 on TCGA-RCC. We rerun it with the same patch encoder as every other method so that feature quality is controlled. The target is to approach its accuracy while reading a small fraction of its high-power pixels, not to exceed it.

The strongest budget-limited competitor is HAG-MIL, run from its released code (BearCleverProud/HAG-MIL). It is the strongest competitor because it solves exactly our problem, spending high-power compute selectively, with square tiles. Its reported figures are 0.946 AUC and 0.874 F1 on Camelyon16, 0.921 AUC on TCGA-NSCLC, and 0.982 on TCGA-RCC. For fairness, we sweep its top-$k$ per level so that its total high-power pixel count matches each of our budgets. It gets the same encoder, the same training schedule, and the same hyperparameter-search budget as our method. Beating it means a strictly higher AUC or grading score at matched budgets in the low-budget regime, plus higher coverage of small lesions, with gains larger than the variation across seeds and folds.

Two ablations isolate the key ingredient. The first swaps FTU masks for SLIC superpixels of matched count, the geometric segmentation used in HACT-Net's tissue graph, and keeps everything else fixed. This tests whether functional units matter or whether any segmentation works. The second is an HGPMIL-style variant (Wang et al.). It keeps grid tiles and HAG-MIL-style allocation but appends FTU-membership and unit-shape features to each tile. This tests whether units should be the instances and allocators, or merely extra features. We also remove the coverage floor ($m=0$) to measure its specific contribution.

Why not just…?

Why not just read the whole slide with CLAM and forget budgets? On tasks like TCGA-RCC this is a real threat: at 0.991 AUC there is little left to gain. However, full-resolution reading dominates the cost of inference, and practical scanning and triage settings are budget-limited. The budget sweep settles this empirically. If CLAM-level accuracy requires close to full reading while our allocator reaches it at a small fraction, the proposal is useful. If tile-based methods with small budgets already match full reading, the allocation problem is not worth solving, and we will report that.

Why not just tune HAG-MIL's top-$k$ larger? A larger $k$ buys coverage with more compute. The question is which method is better at equal compute, and the matched-budget sweep answers it. HAG-MIL's own authors identify the low-attention small lesion as its failure. Our lesion-coverage-by-size measurement on Camelyon16 shows whether the unit floor fixes that failure or merely moves it.

Why not just use superpixels, which need no annotations? This is the most serious alternative, because FTU segmenters need dense masks (Jain et al.) and exist for only five organs. If matched-count superpixels perform as well, the "functional" part of the claim is wrong. We would then reframe the work as segment-based allocation, which is still a useful result. The superpixel ablation is therefore a primary experiment, not a footnote.

Why not just add structural features to grid tiles? HGPMIL shows that priors attached to tiles help. If the feature-augmented grid variant matches unit-as-instance allocation, the gain comes from the information, not from restructuring computation. The ablation separates the two.

Proposed Experiments

PANDA prostate grading (10,616 slides) is the headline task. Prostate is an HuBMAP organ, so the gland segmenter transfers directly. Gleason grading is also defined by gland architecture, which square tiles cut apart. HMIL evaluates on PANDA, but no reported result was found on Lacuna for PANDA, so we will establish all baselines ourselves under a shared protocol.

TCGA-RCC (kidney; glomeruli and tubules) and TCGA-NSCLC (lung; alveoli, where segmentation Dice is only about 0.72) test generality and robustness to imperfect units. Camelyon16 is the stress test. Lymph node is outside the HuBMAP organs, so units come from the class-agnostic segmenter, and its lesion annotations let us measure coverage by lesion size, alongside the localization score DSMIL reports (0.437).

Metrics are AUC, F1, and grading agreement, together with high-power pixels read, peak GPU memory, and wall-clock time, all reported as curves over $B$. Compute cost includes the low-power segmentation pass, so the comparison is net of the part-detection overhead that GAMap warns about.

The decisive check is a single figure: lesion coverage versus lesion size at matched budget for random sampling, HAG-MIL, superpixel units, and FTU units. If FTU units do not raise small-lesion coverage above HAG-MIL's, and do not beat superpixels on PANDA grading at low budgets, the thesis is refuted.