Unlocking Fine-Grained Cell Representations via Soft-Interior Rim Dense-Grid Pooling in Pathology Foundation Models

Wenwen Lou, Mingyi He, Xinyi Guo, Yining Li, Weiming Chen, Jiawen Li, Lianghui Zhu, Zhiwei Yang, Xitong LingarXiv 2026 · arXiv

TL;DR

Soft-Interior Rim Dense-Grid Pooling (SRDP) is a training-free method that extracts high-fidelity features from frozen pathology foundation models (PFMs). By using a dense overlapping grid and multi-scale "rim" pooling, it significantly improves cell classification and retrieval accuracy without requiring any model fine-tuning.

What Problem Does This Paper Address?

Pathology Foundation Models (PFMs)—like UNI, CONCH, and Virchow—have revolutionized computational pathology by providing powerful, pre-trained features for tissue analysis. However, most of these models are Vision Transformers (ViTs) that process images in a fixed grid of patches (typically 16x16 pixels).

This creates a "spatial bottleneck" when analyzing individual cells:

  1. Size Mismatch: A nucleus is often smaller than a single patch.
  2. Boundary Issues: Cells rarely align perfectly with a grid; they often straddle the boundaries of multiple patches.
  3. Feature Mixing: Standard "token averaging" (averaging the features of all patches a cell touches) often mixes the signal from the nucleus with the surrounding cytoplasm and background tissue, blurring the fine-grained details needed for precise cell typing.

Existing solutions either require expensive, task-specific training (like HoVer-Net) or settle for coarse, noisy representations.

Methods

The authors propose SRDP, a "geometry-aware readout" that works on top of any frozen PFM. It consists of three key innovations:

  1. Dense-Grid Extraction: Instead of the standard non-overlapping patch stride, SRDP uses a half-stride. This creates four overlapping lattices, effectively quadrupling the number of feature tokens and providing multiple "views" of the same spatial location.
  2. Soft-Interior Pooling: Rather than a binary "in or out" assignment for tokens, SRDP uses fractional coverage weights based on the cell mask. This ensures that tokens only partially covered by a cell contribute proportionally, preserving the irregular shape of the nucleus.
  3. Rim Pooling: Morphology (the shape and texture of the cell boundary) is critical for diagnosis. SRDP uses morphological erosion to create two "inner rims" (at 2-pixel and 8-pixel depths). It then pools features separately for the whole cell and these two rims, concatenating them into a final descriptor.

Because SRDP is deterministic and requires no training, it can be dropped into existing workflows to immediately boost the performance of frozen encoders.

Datasets

The researchers evaluated SRDP across four diverse datasets and six different frozen PFM encoders:

  • CoNIC & Lizard: Large-scale datasets focused on colorectal nuclei.
  • NuCLS: Breast cancer nucleus classification.
  • MoNuSAC: A multi-organ nuclei segmentation and classification challenge.
  • Encoders tested: DINOv3 (Base and Large), CONCH (v1 and v1.5), Prov-GigaPath, and UNI.

Key Insights and Ideas

  • The Bottleneck is Spatial, Not Semantic: The "aha moment" here is that foundation models already "know" enough to distinguish cell types; the problem is simply how we extract that knowledge from the coarse patch grid.
  • Overlapping Patches are "Free" Information: By simply shifting the patch window (half-stride), you can recover spatial detail that is normally lost, without changing the underlying model weights.
  • Contours are Feature-Rich: By explicitly separating the "rim" from the "interior," the model captures morphological cues (like the lobulated shape of a neutrophil) that get washed out in a global average.
  • Training-Free Generalization: SRDP consistently improved performance across every encoder tested, suggesting it is a universal improvement for the ViT architecture in pathology.

Key Results

  • Accuracy Boost: SRDP improved mean balanced accuracy (BAcc) by 5.68 percentage points over standard token averaging across 24 different settings.
  • Colorectal Excellence: On the CoNIC and Lizard datasets, the gain was even more dramatic, averaging 11.22 points.
  • Few-Shot Performance: In scenarios with very few labeled examples (e.g., 1–10 cells per class), SRDP maintained a massive lead over baselines, proving that its descriptors are inherently more discriminative.
  • Superior Retrieval: When used for "similar-cell retrieval" (finding cells of the same type using only feature similarity), SRDP significantly outperformed other readout methods, concentrating high similarity scores on morphologically matched cells.

Limitations and Future Work

  • Computational Overhead: While training-free, the dense grid extraction is more computationally expensive than a standard stride. At a half-stride, it is roughly 4x slower than native token averaging (though still much faster than a full training run).
  • Fixed Rim Scales: The rim radii ($r=2, r=8$) are fixed in pixel space. While they worked well across the tested datasets, they might need to be scaled if the image resolution (microns per pixel) varies significantly.
  • Dependency on Masks: SRDP assumes that accurate cell masks are already available. Its performance is tied to the quality of the upstream segmentation tool.
  • Future Direction: The authors suggest exploring how these fine-grained representations could be used to model the "neighborhood" or spatial architecture of tissue, moving from cell-level to tissue-level insights.

Summary updated

Abstract

Fine-grained cell representations are essential for characterizing cellular heterogeneity and tissue organization in computational pathology. Pathology foundation models (PFMs) provide powerful pretrained features for transferable tissue analysis. HoVer-Net uses task-trained segmentation and typing, whereas CellViT++ averages native-grid tokens assigned to detected nuclei. However, the former requires cell-specific training; the latter ignores fractional nuclear coverage and can mix nuclear content with neighbouring tissue. Therefore, we propose SRDP, short for Soft-interior Rim Dense-grid Pooling, a training-free geometry-aware readout. We apply the frozen patch projection at half stride, interleave four offset lattices, map the cell mask to fractional coverage weights, and concatenate whole-cell pooling with two inner boundary pools. Across four datasets and six frozen encoders, SRDP improves mean balanced accuracy by 5.68 percentage points over native token averaging across 24 settings, including 11.22 points on average for CoNIC and Lizard. SRDP converts a coarse frozen PFM lattice into a mask-aligned, fine-grained cell representation without updating the encoder.
BibTeX
@misc{lou2026unlocking,
  author = {Wenwen Lou and Mingyi He and Xinyi Guo and Yining Li and Weiming Chen and Jiawen Li and Lianghui Zhu and Zhiwei Yang and Xitong Ling},
  title = {{Unlocking Fine-Grained Cell Representations via Soft-Interior Rim Dense-Grid Pooling in Pathology Foundation Models}},
  year = {2026},
  eprint = {2611.19325},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19325},
}
Figure