C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology

Esha Sadia Nasir, Jiaqi Lv, Gozde N. Gunesli, Shan E Ahmed RazaarXiv 2026 · arXiv

In the rapidly evolving field of computational pathology, "foundation models" have become the new standard. However, most of these models are giants trained on large tissue patches, often missing the fine-grained morphological details of individual cells. When researchers do build cell-centric models, they tend to be computationally massive, making them impractical for analyzing the millions of cells found in a single whole-slide image.

Enter C-LiTE, a lightweight foundation model that proves you don't need a billion parameters to understand a cell. By rethinking how Vision Transformers (ViTs) handle small images, the authors have created a model that is 30x smaller than its peers while delivering state-of-the-art performance.

TL;DR

C-LiTE is a compact, 22-million-parameter foundation model specifically designed for single-cell representation learning in histopathology. By utilizing a native-resolution tokenization strategy and a novel weight-transfer method, it achieves performance comparable to models with 30x more parameters while being significantly faster and more resource-efficient.

What Problem Does This Paper Address?

Computational pathology relies on characterizing individual cells for tasks like tumor grading and immune profiling. Current foundation models face a "scale mismatch":

  1. Tissue-level models (like UNI or Virchow) are trained on large patches. They capture cellular features only implicitly and often require upsampling small cell crops, which adds computational cost without adding new information.
  2. Existing cell-level models (like CytoFormer) are massive, often exceeding 600 million parameters. Running these models on every single cell in a whole-slide image is computationally prohibitive.
  3. Architectural hurdles: Standard ViTs use 16x16 patches. On a small 64x64 pixel cell crop, this yields only 16 tokens—too few for the self-attention mechanism to learn complex spatial relationships.

Methods

The researchers introduced several key innovations to make a small model perform like a giant:

  • Native-Resolution Tokenization: Instead of upsampling, C-LiTE processes 64x64 pixel crops using a 4x4 patch size. This produces a 16x16 grid (256 tokens), providing the "spatial density" needed for effective self-attention at the cellular scale.
  • PI-PatchEmbed (Pseudoinverse-Initialised Patch Embedding): To jumpstart the model with visual knowledge from large-scale pre-training (like ImageNet), the authors needed to map 16x16 patch weights down to a 4x4 size. They used a Moore-Penrose pseudoinverse-based transformation to mathematically "resize" the weights while preserving their spatial response.
  • Hybrid SSL Objective: The model was trained using a combination of DINO (global image-level distillation) and iBOT (masked patch-level prediction) objectives, adapted specifically for cell-centered crops.
The C-LiTE architecture featuring native-resolution adaptation and self-distillation.

Datasets

The authors assembled what is likely the largest publicly available cell-level dataset for self-supervised learning:

  • Pre-training: 60 million cell-centered crops extracted from The Cancer Genome Atlas (TCGA), balanced across 19 cancer types and 16 tissue types.
  • Evaluation: The model was benchmarked across seven public datasets, including MIDOG (mitosis), NuCLS (nucleus classification), PanNuke, CoNIC, and Immunocto (immune cell phenotyping).

Key Insights and Ideas

  • Resolution Trumps Model Size: The biggest performance boost didn't come from adding layers, but from reducing the patch size. Finer spatial granularity allows the model to "see" internal structures like chromatin texture and membrane contours that are lost at coarser resolutions.
  • The "Green AI" Advantage: C-LiTE demonstrates that high-quality representations can be learned with a fraction of the carbon footprint. It achieves a 21.1x higher throughput than CytoFormer, making it a "practical" foundation model for real-world clinical pipelines.
  • Distillation as Context: While C-LiTE is excellent at pure morphology, the authors found that distilling knowledge from tissue-level "teacher" models (like UNI2) helped the model handle ambiguous classes where surrounding tissue context provides vital clues.

Key Results

C-LiTE stands out for its incredible efficiency-to-accuracy ratio:

  • Accuracy: Achieved a mean macro-F1 of 0.670, nearly matching the much larger CytoFormer (0.682) and outperforming tissue-level giants like Virchow2 and UNI2 on cell-centric tasks.
  • Efficiency: It uses 30x fewer parameters and 16.2x fewer FLOPs than CytoFormer.
  • Speed: C-LiTE can process 1 million cells in roughly 16 minutes on a single NVIDIA A100 GPU, whereas CytoFormer takes over 5.5 hours for the same task.
Performance vs. Parameter count: C-LiTE approaches the performance of models 30x its size.

Limitations and Future Work

The authors identify several areas for growth:

  • Linear Probing Only: The study focused on "frozen" features. The impact of full fine-tuning on downstream tasks remains to be explored.
  • Single Resolution: Pre-training was limited to 0.5 microns per pixel (mpp). Future versions could benefit from multi-resolution training to handle different scanner magnifications.
  • Stain Diversity: The current corpus is limited to H&E-stained images from TCGA. Testing on other stains (like IHC) or different laboratory protocols is a necessary next step for clinical validation.

Overall, C-LiTE represents a shift toward "sensible" AI in pathology—prioritizing architectural fit and computational efficiency over raw parameter count.

Summary updated

Abstract

Characterising individual cells is fundamental to computational pathology, as cellular morphology informs tasks such as tumour grading, mitotic assessment and immune profiling. However, most pathology foundation models are trained on tissue patches and capture cellular features only implicitly, while existing cell-level models are either restricted to specific cell populations or rely on computationally expensive architectures with hundreds of millions of parameters. Applying ViTs directly to small cellular crops also presents an architectural challenge: upsampling increases computational cost without adding morphological information and may introduce interpolation artefacts. To address these limitations, we introduce C-LiTE, a lightweight foundation model for cellular representation learning in H&E-stained histopathology, with only 22 million parameters. C-LiTE employs a native-resolution tokenisation strategy tailored to cellular inputs, processing 64 x 64 px cell-centred crops using a 4 x 4 patch embedding to produce a 16 x 16 token grid without image upsampling. To transfer visual knowledge from large-scale pre-training, we adapt the patch embedding weights of a pre-trained ViT-5-S using a pseudoinverse-based transformation, enabling weight initialisation at the finer patch resolution. We further assemble a pre-training dataset of 60 million cell-centred crops from TCGA, balanced across 19 cancer types and 16 tissue types. To our knowledge, this represents the largest public dataset for self-supervised pre-training on H&E images at the cellular level. We evaluate C-LiTE on seven public datasets spanning mitotic figure recognition, nucleus and cell-type classification, and immune cell phenotyping. Under frozen-feature evaluation, C-LiTE achieves a mean macro-F1 of 0.670, approaching CytoFormer (0.682) with approximately 30x fewer parameters.
BibTeX
@misc{nasir2026c,
  author = {Esha Sadia Nasir and Jiaqi Lv and Gozde N. Gunesli and Shan E Ahmed Raza},
  title = {{C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology}},
  year = {2026},
  eprint = {2611.26316},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26316},
}
Figure