A whole-slide pathology image is too large for a neural network to see at once. It is cut into thousands of small tiles, each tile is turned into a feature vector, and a pooling step merges the vectors into one slide-level prediction. That pooling step usually ignores where each tile sat on the slide, and this matters because a cell's meaning depends on its neighbors. Methods that restore context split into two camps with opposite failures. Global, low-rank mixers compress all tiles through a small shared summary, and they blur small, rare regions into the surrounding tissue. Local, sparse mixers connect each tile only to nearby tiles, and they break a single lesion into disconnected fragments while missing long-range structure. Subspace clustering has studied exactly this trade-off. When a nuclear-norm (low-rank) penalty and an $\ell_1$ (sparsity) penalty are placed on the same matrix that explains each point as a combination of the others, the result can keep each group internally connected and different groups apart.
We propose to make this joint objective the context layer of multiple instance learning (MIL), the setting where a slide is treated as an unordered bag of tiles with only a slide-level label. The one new ingredient is a sparsity penalty that is weighted by physical distance on the slide. For each slide, a few unrolled solver steps compute a patch-to-patch coefficient matrix in which each tile is rebuilt from a small number of other tiles. Nearby tiles are cheap to use, distant tiles are expensive but allowed, and a low-rank term keeps tiles of the same tissue type linked across the whole slide. The coefficients then carry features between tiles, and they double as an interpretable map of which tiles explain which. The central prediction can be checked directly: on CAMELYON16, the gains should appear on slides with small metastases, and coefficient mass crossing annotated tumor boundaries should fall well below that of softmax or low-rank mixers.
Core Claim
The over-smoothing versus fragmentation problem in spatial MIL is the same problem as the connectivity versus separation trade-off in subspace clustering, and the same remedy applies. Provable Subspace Clustering: When LRR meets SSC (LRSSC) supplies both the diagnosis and the method. It proves that pure low-rank representation leaks across subspaces unless those subspaces are independent, and that pure sparse representation fragments clusters under skewed sampling. It also shows that an intermediate weight on the $\ell_1$ term reaches a "sweet spot" that keeps each point's coefficients inside its own subspace while leaving each cluster connected. A tiny metastasis is a textbook skewed minority subspace: a few dozen tumor tiles among ten thousand normal ones. We claim that adding the spatial structure of the slide, as a distance weight on the $\ell_1$ term, moves this sweet spot into a regime that is both more accurate and easier to interpret than existing context aggregators.
For one slide with tile features $X \in \mathbb{R}^{d \times N}$ (one column per tile, $N$ tiles) and tile coordinates $p_i$, the per-slide problem is
$$\min_{C,\,S}\; \|X - A C^\top - X S\|_F^2 + \alpha \|C\|_* + \beta \|C\|_1 + \gamma \sum_{i,k} w_{ik} |S_{ik}|,$$
The symbols are as follows. $A \in \mathbb{R}^{d \times m}$ holds $m$ anchor tiles, about 256, chosen by spatially stratified k-means. $C \in \mathbb{R}^{N \times m}$ gives each tile's coefficients on the anchors, and it carries the low-rank and entrywise-sparse penalties exactly as in LRSSC. $S$ is supported only on each tile's spatial neighbors within radius $r$, with zero diagonal. $w_{ik} = \exp(\|p_i - p_k\|/\tau)$ makes distant neighbors costlier. $\alpha, \beta, \gamma, \tau$ are learned per unrolled layer. The anchor form keeps nuclear-norm thresholding at $O(Nm^2)$ per step instead of the $O(N^3)$ that makes classical solvers impractical, which is the scalability limit named in direction 1. Because both penalties act on the same $C$, the separation property survives, unlike a plain "low-rank plus sparse" sum. The affinity between tiles $i$ and $j$ is $(CC^\top)_{ij} + |S_{ij}|$. It is a pairwise object a pathologist can inspect, which Exploring Low-Rank Property in Multiple Instance Learning for Whole Slide Image Classification names as the thing its latent bottleneck cannot provide.
Relation to Prior Work
Three works motivate the proposal by exposing one side of the trade-off. ILRA-MIL, in the low-rank paper above, routes all tiles through a learned latent matrix to get linear cost. It has no sparsity term and does not use coordinates. PSA-MIL adds distance-decayed priors to attention and prunes weak interactions, which gives spatial sparsity but no low-rank global structure. GDS-MIL runs graph attention on an 8-neighbor tile graph and reports that stacking layers over-smooths, which is a negative result that rules out "just go deeper on the local graph." SparseConvMIL places features on the real coordinate grid. Its downsampling factor forces a choice between isolated tiles and tile collisions. Our radius-based support for $S$ uses true distances, so it avoids that choice.
Two works supply or constrain the mechanism. Scatterbrain approximates softmax attention as a sparse part plus a low-rank part and shows lower variance than low-rank alone. It is the closest structural relative. It differs in two ways. It approximates softmax, whose rows are positive and sum to one, so every token always mixes with something. Its sparse support comes from feature hashing, not physical space. TransMIL already pairs a low-rank Nyström attention with local convolutions on a pseudo-grid. That makes it the design a skeptic will say we are reinventing, so we fix its grid and test it head-on. The tensor methods Sturm and Multilinear Regression for Embedded Feature Selection show that pairing nuclear and $\ell_1$ penalties preserves spatial structure in brain imaging. They regularize model weights, not per-sample interaction graphs, and they are not trained end to end.
Research Program
The first stage asks whether the phenomenon exists in real pathology features before any model is built. On CAMELYON16 tumor slides with lesion masks, we subsample bags of up to 2,000 tiles so that exact LRSSC with full singular value thresholding is affordable. We then sweep the $\ell_1$ weight and measure two quantities. Leakage is the fraction of coefficient mass that crosses the tumor/normal boundary. Lesion connectivity is the number of connected components that a single annotated lesion splits into after thresholding the affinity graph. This checks the separation guarantee and the empirical connectivity result of LRSSC on this data. It also shows whether distance weighting moves the sweet spot. If pure low-rank shows no leakage beyond joint low-rank and sparse on these features, the premise fails and we stop.
The second stage builds the MIL layer. Each forward pass runs $K = 3$ to $5$ unrolled proximal-gradient steps on the objective above. The penalty weights are learned from slide labels, which bears on the hyperparameter-tuning problem that direction 1 leaves open. The contextualized features $X + \lambda(AC^\top + XS)$ feed an attention pooling head. Backpropagating through singular value decomposition is unstable when singular values are close together. We will compare truncated-SVD gradients with gradient stopping on the singular vectors and report which is stable. The third stage turns the affinity into explanations. Spectral clustering of $CC^\top + |S|$ gives tissue regions, and lesion-level heatmaps come from the attention head.
Encoders stay frozen and match each baseline's reported setup, so that gains can be credited to aggregation. Tile extraction and coordinates follow the CLAM pipeline as implemented in HKU-MedAI/ConSlide.
Baselines
The naive reference is attention MIL with instance clustering. We use CLAM from Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images, which scores each tile independently. It reports 0.953 AUC on CAMELYON16 in its own paper, while the ILRA paper reports CLAM-MB at 87.79% AUC. The gap between those two numbers is a reminder that protocol matters, so we rerun CLAM on our splits rather than quote either figure.
The strongest existing method on this question is ILRA-MIL with the pathology-specific low-rank contrastive pretraining (LRC), which reports 96.49% AUC on CAMELYON16 and 97.63% AUC on TCGA-NSCLC. It is the strongest competitor because it is the pure low-rank end of our trade-off, already tuned for pathology. Its headline number combines a specialized encoder with the aggregator. For a fair test we run the ILRA-MIL head and our head on the same features, first on ImageNet ResNet50 features and then on LRC features if they can be reproduced. Both get the same number of epochs and the same hyperparameter search budget, which we fix at 30 trials per method. Beating it means a higher mean AUC over five seeds with non-overlapping bootstrap intervals on the same features. On PANDA, ILRA reports Cohen's kappa 0.6562 while the pyramid transformer in Accounting for Dependencies in Deep Learning Based MIL reports quadratic weighted kappa 0.932. These are different metrics, so we report both metrics for every method rather than compare across papers.
TransMIL is run twice: as published, with 93.09% AUC reported on CAMELYON16, and with its pseudo-grid positional encoding replaced by convolutions on true coordinates. The second version is the fair "low-rank global plus local spatial" competitor. PSA-MIL is run with its released spatial priors. It reports gains over MSA-MIL on TCGA-CRC (+5.5% AUC) and TCGA-STAD (+3.1% AUC); no reported CAMELYON16 absolute result was found on Lacuna for PSA-MIL.
The key-ingredient ablation is a Scatterbrain-style hybrid. It uses softmax attention restricted to the same spatial neighborhoods plus a low-rank global attention term, with identical anchors, radius, and parameter count. It isolates one variable: self-expression with $\ell_1$, which can set cross-tissue coefficients exactly to zero, versus softmax mixing, which cannot. Further ablations set $\alpha = 0$ (pure sparse), $\beta = 0$ (pure low-rank), and $w_{ik} \equiv 1$ (no spatial weighting). The proposal wins only if the full model beats all four ablations on leakage and on small-lesion sensitivity. Slide AUC alone is not enough.
Why not just…?
Why not just fix TransMIL's positional encoding to use real coordinates? This is the most serious threat. Low-rank global attention plus local convolutions on the true grid may capture most of the benefit. Our claim is narrower: the Nyström softmax rows are dense and positive, so tumor tiles always absorb some normal-tissue signal, and this should show up as higher leakage and lower small-lesion sensitivity. If the coordinate-fixed TransMIL matches our leakage and small-metastasis sensitivity, then the self-expression machinery is not worth its cost, and we will report that.
Why not just use a Scatterbrain-style sparse-plus-low-rank attention with a spatial sparse part? It is cheaper to implement and is the obvious combination of Scatterbrain and PSA-MIL. It is built in as our key ablation with matched capacity. The distinction we test is that its low-rank component carries no sparsity penalty, which is exactly the regime where LRSSC shows cross-subspace leakage. If it ties us on the leakage metric, the joint penalty on a single matrix is not doing the work.
Why not just use ILRA-MIL with a better encoder? Encoder quality may dominate aggregation, and ILRA's 96.49% already leaves little room in slide AUC. That is why we run both heads on the same features, and why our main endpoints are lesion-stratified sensitivity and pairwise interpretability, which ILRA cannot provide by construction.
Why not just use a shallow spatial graph network? GDS-MIL's own over-smoothing report limits depth, and an 8-neighbor graph cannot link the same tissue type across a slide. Our $\beta = 0,\ \alpha$-only and $\alpha = 0$ ablations bracket this alternative, and the lesion-connectivity metric tests fragmentation directly.
Proposed Experiments
Slide classification uses CAMELYON16, TCGA-NSCLC, and PANDA with the protocols of the cited papers. We report AUC and accuracy, plus both kappa variants on PANDA, over five seeds. CAMELYON16 lesion masks support the decisive checks. The first is slide sensitivity stratified by largest-lesion size, where we predict the largest gains in the smallest-lesion group. The second is the leakage ratio of each method's tile-interaction matrix across annotated boundaries; for attention models this is the attention matrix, and for ILRA it is approximated through the latent bottleneck. The third is the number of connected components per lesion, where the ideal value is one. The fourth is tile-level localization AUC. We also report wall-clock time and peak memory at $N = 10^4$ and $N = 5 \times 10^4$ tiles against TransMIL and ILRA-MIL.
The proposal is supported if the full model lowers leakage relative to every softmax and low-rank competitor and keeps lesions connected where the pure-sparse ablation fragments them, and if this translates into higher small-lesion sensitivity at matched slide AUC or better. It is refuted if the coordinate-fixed TransMIL or the hybrid-attention ablation matches it on these lesion-level measures.
References
- Provable Subspace Clustering: When LRR meets SSC
- Exploring Low-Rank Property in Multiple Instance Learning for Whole Slide Image Classification
- PSA-MIL: A Probabilistic Spatial Attention-Based Multiple Instance Learning for Whole Slide Image Classification
- Enhancing PFI Prediction with GDS-MIL: A Graph-Based Dual Stream MIL Approach
- SparseConvMIL: Sparse Convolutional Context-Aware Multiple Instance Learning for Whole Slide Image Classification
- Scatterbrain: Unifying Sparse and Low-rank Attention
- TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification
- Sturm: Sparse Tubal-Regularized Multilinear Regression for fMRI
- Multilinear Regression for Embedded Feature Selection with Application to fMRI Analysis
- Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images
- Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging