Whole-Slide Spectral Filters on Sparse Tissue Graphs

A whole-slide pathology image is billions of pixels. Models therefore cut it into tens of thousands of small tiles, embed each tile with a frozen network, and reason over the resulting set. To let distant tiles inform each other, current methods approximate. They either compress the slide into a few soft clusters or use low-rank "landmark" attention, which summarizes all tiles through a few hundred representatives. Both shortcuts blur the per-tile maps a pathologist needs to see where a small metastasis sits. The authors of CAMIL say this directly: their landmark approximation "might slightly limit" localization fineness.

This proposal argues the shortcut is unnecessary. A tissue graph, with one node per tile and edges only to the eight touching tiles, is extremely sparse. On sparse graphs, a spectral filter (a reweighting of a signal's smooth versus rapidly varying spatial components) can be computed exactly as a polynomial. That costs only repeated multiplications by a sparse matrix, linear in the number of edges (Defferrard et al.). If the polynomial degree is in the hundreds, each tile can draw context from hundreds of tiles away while keeping its own output. In two sentences: replace landmark attention and cluster pooling with a learned, high-degree polynomial filter run on the full, unpruned 8-neighbor tissue graph. This gives whole-slide context at tile resolution, with a cost and an approximation error we can state in advance.

Core Claim

Let $L$ be the symmetric normalized Laplacian of a slide's 8-neighbor tile graph. $L$ is a matrix that measures how much a signal differs between touching tiles, and its eigenvalues lie in $[0,2]$, so $\tilde L = L - I$ has spectrum in $[-1,1]$ with no eigenvalue estimation needed. Let $X \in \mathbb{R}^{n \times d}$ be the tile features. The layer computes

$$Y = \sum_{k=0}^{K} \theta_k\, T_k(\tilde L)\, X,$$

where $T_k$ is the $k$-th Chebyshev polynomial, evaluated by its three-term recurrence. The coefficients $\theta_k$ are not free parameters. They come from a small learned frequency response $h(\lambda)$, a function of the eigenvalue $\lambda$ with a few parameters per channel, read off by Chebyshev interpolation. This keeps degrees of $K = 128$ to $256$ trainable.

The cost is $O(K|\mathcal{E}|d)$, where $|\mathcal{E}| \approx 4n$ is the number of edges. Information travels at most $K$ hops, so $K$ is chosen on the scale of the tissue's diameter in tiles.

Two facts make this practical. First, the feature dimension can be small. Exploiting Low-Dimensional Manifold of Features measured an effective rank of about 30 for foundation-model tile features, and showed that trainable full-rank projections distort that geometry in low-data regimes. We therefore filter in a $d \approx 32$–$64$ space reached by their frozen-anchor-plus-low-rank projection. That paper acts as a constraint on our design, not as a method we compete with. Second, the expected memory is small. With $n = 50{,}000$ tiles, $K = 256$, and $d = 64$, the stored recurrence states take about 3 GB, and gradient checkpointing reduces this further.

Because $h$ is smooth on $[-1,1]$, Chebyshev truncation error falls geometrically in $K$. This is a standard approximation-theory result for analytic functions, and it means the "approximation trade-off" can be measured and bounded rather than accepted.

The output is one feature vector per tile. A tile-level classifier turns it into a full-resolution evidence map, and attention pooling turns that map into the slide prediction. The learned responses $h$ are themselves readable. They show which spatial frequencies the model uses: sharp, high-frequency contrasts at tumor boundaries versus slow, low-frequency variation across tissue architecture.

Relation to Prior Work

The closest prior work is Representation Learning of Histopathology Images using Graph Neural Networks. It applies ChebNet to whole-slide images, reaching 88.8% accuracy and 0.89 AUC on TCGA lung subtyping. However, it runs the filters on a fully connected learned graph over a random sample of tiles. A dense graph removes both reasons to use polynomial filters: each multiplication costs $O(n^2)$, and the $K$-hop locality collapses to one hop. We do the opposite: a strictly sparse physical graph, every tile kept, and a degree roughly 30 times higher than their Cheb-7.

GTP also builds an 8-neighbor graph. It then applies a few graph convolutions and mincut pooling into soft clusters, which can merge a micro-metastasis into its surroundings. We never coarsen the graph.

CAMIL and HAG-MIL get global context from Nyström low-rank attention. CAMIL also attends locally over 8-neighbors. Our layer is meant as a direct replacement for the Nyström global branch. The rest of these architectures stays the same, which makes the comparison clean.

Two scalable spectral methods were considered as building blocks. BernNet supplies the idea of parameterizing a filter by its response. Its nonnegativity constraint rules out the signed, boundary-enhancing responses we want, so we use Chebyshev interpolation. Coreset Spectral Clustering and PEARL offer coarsening and linear-time positional encodings respectively. We treat them as competing routes to the same goal, not as parts of the method.

Research Program

We will build the graph at 20x magnification per tissue-connected component. We will implement the filter layer with batched sparse matrix multiplication and checkpointed recurrences, and pair it with a tile classifier and gated attention pooling.

All models will share one frozen patch encoder: a pathology foundation model of the kind used in the manifold paper. This matters because published numbers in the file mix SimCLR, ImageNet, and foundation-model features. We rerun every baseline rather than compare across papers.

The analysis has three parts. First, a degree sweep. If long-range context matters, performance should rise until $K$ approaches typical tissue diameter on subtyping tasks, and should level off early on metastasis detection. Second, an approximation-error audit. On small slides where exact eigendecomposition is feasible, we compare the truncated filter to the exact $U h(\Lambda) U^\top X$, where $U$ and $\Lambda$ are the eigenvectors and eigenvalues of $L$. Third, a study of what the learned responses look like across tasks.

Baselines

The first naive reference is plain attention-based multiple instance learning (ABMIL), which pools tiles with learned weights and ignores position. On CAMELYON16, DSMIL reports 84.5% accuracy for it. A second naive reference, which tests our main ingredient, is the same model with a fixed heat-kernel low-pass filter at the same degree. This is roughly "smooth the features over the tissue." If a fixed filter matches a learned one, the learned frequency response is not doing the work.

The strongest localization competitor is CAMIL, rerun with our features. It reports a CAMELYON16 Dice score of 0.515, against 0.459 for CLAM-SB and 0.103 for TransMIL (CAMIL). It also reports AUC of 95.9% on CAMELYON16, 88.1% on CAMELYON17, and 97.5% on TCGA-NSCLC. CAMIL is the strongest competitor because it already combines 8-neighbor local context with linear-time global context. Any gain we show must therefore come from removing the landmark approximation, not from adding spatial structure.

For detection and the FROC localization score, the strongest reported system is Snuffy, with AUC 0.987 and FROC 0.675 on CAMELYON16. For kidney subtyping it is HAG-MIL, with AUC 0.982 on TCGA-RCC (HAG-MIL). We will also run GTP, as the cluster-pooling representative, and Adnan et al.'s dense-graph ChebNet, as the direct predecessor.

For fairness, every method uses identical features, splits, five-fold cross-validation, and the same hyperparameter search budget of 30 trials each. Wall-clock time and peak GPU memory are logged per slide.

The key ablation is "CAMIL with its Nyström branch swapped for our filter," with everything else fixed. Beating CAMIL means three things under matched features: a Dice improvement beyond the fold-to-fold standard deviation, FROC at least matching Snuffy's rerun, and slide AUC no worse than CAMIL's.

Why not just…?

Why not just give Nyström attention more landmarks? This is a real threat. If the localization loss comes only from too few landmarks, raising the count may close the gap at acceptable cost. We will sweep landmark counts up to the memory limit and plot Dice against compute for both methods. The proposal succeeds only if the filter dominates that curve, not merely the default setting.

Why not just stack graph convolution or attention layers on the 8-neighbor graph? A linear degree-$K$ polynomial is mathematically a $K$-layer linear graph convolution. Spatial networks can therefore reach the same range in principle. In practice, a 200-layer network stores 200 sets of parameters and nonlinear activations, and it tends to average all tiles together ("oversmoothing"). Adnan et al.'s head-to-head comparison also favored ChebNet over GraphSAGE (Representation Learning...). We test this directly with depth-matched and compute-matched GCN and GAT baselines at 8, 32, and 128 layers.

Why not just add scalable positional encodings and keep a linear transformer? Random-initialization encodings from PEARL scale linearly and help on 38-million-node graphs. However, they still feed an attention layer that must itself be approximated at slide scale, so the localization question stays open. We will run R-PEARL encodings with Nyström attention as an additional baseline. If that baseline matches our Dice, the filter idea reduces to "better position information" and we would say so.

Why not just smooth the attention heatmap afterward? Post-hoc smoothing is a fixed low-pass filter applied after training. It cannot sharpen boundaries or change which tiles are learned to be evidence. The fixed-heat-kernel ablation, plus a version that smooths after training, measures exactly this.

Proposed Experiments

The decisive experiment is CAMELYON16 with matched features. We report Dice against pathologist annotations, FROC, and slide AUC for our model, CAMIL, CAMIL with the swapped-in filter, the landmark sweep, Snuffy, and the depth-matched GCN.

Generalization and range are tested on CAMELYON17 (multi-center AUC) and on TCGA-NSCLC and TCGA-RCC subtyping. On the subtyping tasks, the degree sweep should show whether long-range context, rather than local smoothing, drives accuracy.

Scalability is reported as time and memory versus tile count on the largest slides. The goal is linear growth with no tile pruning.

Few-shot robustness is checked at 8 shots on CAMELYON16 with and without the low-rank projection. The comparison point is the manifold paper's ABMIL+MR, which rose from 49.2 to 56.5 AUC. This tests whether filtering preserves that geometric benefit.

The note fails cleanly if the learned filter is matched by a fixed heat kernel, by a large-landmark Nyström model, or by deep spatial convolutions at equal compute.