Robust Residual Recall for Multi-Scale Whole-Slide Pathology

Fast whole-slide pathology models look at a slide at low magnification, use a learned attention score to guess where the interesting tissue is, and zoom in only there. PathCTM reaches 93.1% AUC on BRACS-3 while inspecting about 4% of patches. The weak point is well documented. If a tiny cluster of malignant cells shows no contrast at low magnification, the attention score never flags it, and it is discarded for good. This proposal adds a second, label-free route into the zoom list. We split the low-magnification feature map of each slide into a low-rank part, meaning a few repeating tissue patterns that explain most of the slide, and a sparse part, meaning the few places that do not fit those patterns. Places with large sparse residuals get inspected at high magnification even if the attention score ignores them.

The split is computed with the alternating direction method of multipliers (ADMM), an optimizer that handles a problem with several competing penalties by cloning variables and alternating cheap closed-form steps, one per penalty. It is the standard tool for "low-rank plus sparse" decomposition. It also comes with error bounds that control the sparse part separately from the low-rank part. We turn those bounds into a principled zoom threshold, replacing a fixed top-k budget. The claim is modest and testable. At a matched patch budget, a residual-based recall channel catches lesions that attention-based pruning misses, and it does so better than simpler outlier scores. If the simpler scores do just as well, we will say so and drop the ADMM machinery.

Core Claim

Attention-based pruning and residual-based recall fail in different ways, because they answer different questions. The attention score is trained on slide labels, so it learns "what looked diagnostic at low resolution in training." The residual is label-agnostic and asks "what does not look like the rest of this slide." A small tumor nest with weak low-magnification contrast can still shift the low-resolution feature vector off the slide's dominant tissue patterns, even when that shift does not resemble the tumor examples the attention head was trained on.

The robust split matters beyond simply measuring distance from the slide's average for one specific reason. On slides with large tumor burden or many tissue types, an ordinary principal-component fit absorbs the abnormal tissue into its subspace. That hides the remaining small lesions. A robust decomposition is designed to resist this contamination.

Formulation

Let $F$ be the matrix of low-magnification patch features for one slide, with one row per tile ($n$ tiles) and one column per feature channel ($C$ channels) from a frozen encoder. We solve

$$ \min_{L,S}\; \tfrac12\|F-L-S\|_F^2+\lambda_*\|L\|_*+\lambda_s\sum_{i=1}^{n}\|S_{i\cdot}\|_2 , $$

where $L$ is the low-rank tissue background, $\|L\|_*$ is its nuclear norm (the sum of singular values, which encourages low rank), $S_{i\cdot}$ is the residual row for tile $i$, and $\lambda_*,\lambda_s$ are penalty weights. We penalize whole rows rather than individual entries, as a group-sparsity penalty, because a lesion tile deviates across many channels at once. In an optional variant, we also arrange tiles on their spatial grid as a tile-row × tile-column × channel tensor, so a smoothness penalty can discourage isolated one-tile noise.

ADMM splits the problem into singular value thresholding for $L$ and row-wise shrinkage for $S$, and both steps have closed forms. The zoom rule sends a tile to high magnification if it falls in the attention top-k or if its residual norm $\|S_{i\cdot}\|_2$ exceeds a threshold. The threshold is set from the noise level estimated on held-out normal slides, scaled by the separate sparse-error bound.

Relation to Prior Work

The method comes from Robust Tensor Decomposition with Gross Corruption. That paper separates a low-rank tensor from sparse gross corruption with ADMM and bounds each component's error separately. It treats the sparse part as noise to remove; we treat it as the signal that decides where to look. We also swap its entrywise sparsity for tile-wise group sparsity.

Per-iteration singular value decompositions are the known bottleneck for this family of methods. For scaling, we borrow two remedies: the core-tensor reduction from Generalized Higher-Order Orthogonal Iteration for Tensor Decomposition and Completion, and the data-adaptive projection from Fast Guaranteed Tensor Recovery with Adaptive Tensor Nuclear Norm. The latter reports 38.06 PSNR versus 30.01 for standard tensor nuclear norm on WDC with 95% missing entries, and 5–10× speedups.

The motivation comes from two papers. Thinking in Scales prunes by low-magnification attention and names the irreversible-pruning failure; our proposal keeps its pruning and adds a recall channel. Exploring Low-Rank Property in Multiple Instance Learning shows that histology patch features cluster in low-rank subspaces. It uses this as a gradient-trained regularizer inside the classifier. We use the same observation outside the classifier, as a recall mechanism, and never ask the low-rank structure to be discriminative.

Agentic Visual Reasoning via Active Perception learns where to zoom with a reinforcement-learning navigator. That navigator still decides from what it can see at coarse scale, which is the same blind spot. Cello shows that micro-lesion hit rate under a budget is measurable: 72.5% at the top 10% of patches on stomach slides, versus under 30% for baselines. We adopt its budgeted-inspection protocol as our primary metric.

Research Program

We build three components.

The first is a slide-level robust decomposition module. It runs on frozen low-magnification features at 1.25×–2.5× and outputs a residual map in seconds per slide.

The second is a two-channel zoom policy on top of the PathCTM pipeline. The budget is split between attention top-k and residual recall, and the split is tuned on validation data.

The third is an analysis of when robustness matters. We stratify slides by tumor burden and by the number of tissue types present. This lets us test whether the gap over plain principal-component residuals grows where contamination is expected. We also test whether the bound-derived threshold gives stable recall across slides, compared with fixed top-k.

Baselines

The naive upper reference is exhaustive high-resolution inference with CLAM, which inspects every patch. PathCTM reports this at 92.0% AUC on BRACS-3. It tells us what is lost by pruning at all. A second reference is PathCTM itself, the strongest pruning method in the file, at 93.1% BRACS-3 AUC with about 4% of patches (Thinking in Scales). We rerun it with the same frozen encoder and splits.

On CAMELYON16, where small metastases make the recall question sharpest, the strongest reported classifier is LRC + ILRA-MIL at 96.49% AUC (Exploring Low-Rank Property), ahead of TransMIL at 93.09%. It is included as the full-inspection accuracy target. We found no reported result on Lacuna for PathCTM on CAMELYON16, so we produce that number ourselves.

All budgeted methods get the same total patch count (4% and 10%), the same features, and the same hyperparameter search budget. Beating PathCTM means higher lesion-level recall and equal-or-better slide AUC at identical budget, with wall-clock overhead under 10%. Reaching exhaustive-inference AUC at a 10% budget would be a strong result.

The key ablations replace the robust residual while keeping the budget split fixed. The replacements are uniform random extra tiles, ordinary principal-component reconstruction residual with the same rank, k-nearest-neighbour feature distance within the slide, and entrywise rather than group sparsity.

Why not just…?

Why not just keep more patches, or add random exploration? Raising PathCTM's budget recovers some missed lesions, but at linear cost. Random extras spend most of the budget on normal tissue, since lesions occupy a tiny fraction of a slide. The random-tile ablation at matched budget measures exactly how much of our gain is "more looking" versus "looking in the right place."

Why not just use a plain PCA or nearest-neighbour outlier score, without ADMM? This is the real threat. For slides with one tiny lesion in homogeneous tissue, ordinary principal-component residuals may perform identically, because a few tiles barely move the fitted subspace. The robust formulation should win only where abnormal tissue is plentiful enough to contaminate the fit, which is the regime Robust Tensor Decomposition with Gross Corruption targets. The burden-stratified comparison settles this. If the gap is negligible across strata, we report the simpler score as the contribution and drop ADMM.

Why not just train the attention head better, for example by distilling from high resolution? Resolution-Based Distillation shows that low-magnification students can approach high-magnification teachers, staying within 3% on LUAD at 1.25×. That would sharpen the attention score, and we include a distilled-attention baseline. But a better-trained attention head is still a label-trained score, and it still misses lesion appearances underrepresented in training. We therefore measure recall separately on lesions the distilled attention misses, to check whether the residual channel is complementary.

Proposed Experiments

We evaluate on CAMELYON16 and BRACS (3- and 7-class), with TCGA-NSCLC as a sanity check. The metrics are slide AUC, lesion-level recall of inspected high-magnification tiles following Cello's budgeted hit-rate protocol, patches inspected, and wall-clock time.

The decisive check is recall on the subset of annotated metastases smaller than a few tiles that PathCTM's attention prunes. If the residual channel recovers a substantial fraction of these at no loss in AUC, the core claim holds. If it recovers them no better than random extras, the claim fails.

A controlled test pastes small high-magnification tumor regions into normal slides, with their low-magnification appearance generated by true downsampling. We vary lesion size and background tumor burden, and plot the recall of the robust residual against the plain principal-component residual.

A final check asks whether the bound-derived threshold gives lower slide-to-slide variance in recall than fixed top-k at the same average budget. This is the one place where the theory is supposed to pay off in practice.