EviSlide-MIL: Multi-Query Pooling and Shared Prediction for Class-Incremental Whole-Slide Image Classification

Zhouhan Qian, Sicheng Chen, Chad Wong, Tianyi Zhang, Enhui Chai, Zeyu Liu, Fei XiaarXiv 2026 · arXiv

In the rapidly evolving field of computational pathology, we often face a practical dilemma: how do we teach an existing AI model to recognize new cancer subtypes without it "forgetting" the ones it already knows? Retraining from scratch every time a new diagnostic category is introduced is often impossible due to data privacy constraints and the sheer storage cost of gigapixel Whole-Slide Images (WSIs).

The authors of EviSlide-MIL propose a clever architectural solution to this class-incremental learning (CIL) challenge, ensuring that as models grow to accommodate new knowledge, their old expertise remains rock-solid.

TL;DR

EviSlide-MIL is a class-incremental framework for whole-slide image classification that mitigates catastrophic forgetting by decoupling class-specific evidence aggregation from a shared prediction module. By freezing learnable queries for old classes and using a shared, parameter-efficient predictor, it achieves state-of-the-art performance on tumor subtype classification with minimal model growth.

What Problem Does This Paper Address?

Histopathology slides are massive—often billions of pixels. To analyze them, researchers use Multiple Instance Learning (MIL), where a slide is treated as a "bag" of small tissue patches. A model must learn to aggregate features from these patches to make a single slide-level diagnosis.

The problem arises when we want to add new tumor subtypes sequentially. Standard MIL models suffer from two types of "drift" during incremental learning:

  1. Attention Drift: Updating the model to find new tumor patterns often changes how it "looks" at old ones, degrading the quality of evidence for previously learned classes.
  2. Prediction Shift: Even if the evidence is preserved, the final classification layer becomes biased toward the most recently seen classes, leading to "catastrophic forgetting."
Fig. 1: Motivation. (a) Class-incremental WSI classification with limited replay. (b) Updating attention can alter aggregated evidence. (c) Updating the shared predictor can change class scores.

Methods

EviSlide-MIL introduces a decoupled architecture designed for stability. Its core components include:

  • Multi-Query Evidence Pooling: Instead of a single global attention mechanism, each class is assigned $M$ learnable queries (the paper uses $M=4$). These queries interact with frozen patch features (extracted via foundation models like GigaPath) to pull out class-specific evidence.
  • The "Freeze" Strategy: This is the secret sauce. Once a class is learned, its queries are frozen. This ensures that the attention patterns for that class never change, effectively locking in the "evidence" extracted for old subtypes.
  • Shared Prediction Module: To keep the model compact, all classes share a single projection layer and scoring vector. Adding a new class only requires adding its specific queries and a single bias term, keeping parameter growth linear and extremely low.
  • Knowledge Retention: The framework employs Experience Replay (keeping a tiny buffer of features from old slides) and Knowledge Distillation (ensuring the new model's predictions on old classes match the previous version of the model).
Fig. 2: Overview of EviSlide-MIL. (a) Four class-specific queries aggregate frozen WSI patch features. Shared projection enables classification across all seen classes. (b) Multi-query pooling extracts patch-level evidence through attention-weighted aggregation. (c) Experience replay and knowledge distillation from a frozen teacher support retention during incremental learning.

Datasets

The researchers evaluated EviSlide-MIL using eight tumor subtypes from The Cancer Genome Atlas (TCGA), organized into four two-class stages:

  • Lung: LUAD, LUSC
  • Breast: IDC, ILC
  • Renal: KIRC, KICH
  • Esophageal: EAC, ESCC

They tested the model using features from state-of-the-art pathology foundation models, including GigaPath and CONCH.

Key Insights and Ideas

  • Stability Over Complexity: One of the most surprising findings was that mean pooling of query results outperformed more complex learned attention weighting for final aggregation. In an incremental setting, the simplicity of mean pooling provides a "stability property" that prevents the model from over-optimizing on new data at the expense of the old.
  • Multi-Query Diversity: By using four queries per class, the model naturally learns to look for different morphological features. As seen in Figure 4, different queries might focus on central tumor regions versus broader tissue organization, capturing the heterogeneity of cancer more effectively than a single query.
  • Decoupling Works: By separating evidence gathering (queries) from decision making (shared predictor), the authors could freeze the former to stop representation drift while allowing the latter to adapt to an expanding set of candidate classes.
Fig. 4: Four queries in LUSC and LUAD validation cases. Both cases retain full 4,096-anchor normalization and a shared asinh display scale. Case B displays 4,016 anchors in a tissue-selected ROI without renormalization. Red rings mark unmodified maxima.

Key Results

  • Superior Performance: Using GigaPath features, EviSlide-MIL reached a 77.80% Final F1 score, significantly outperforming established methods like CLAM-SB + AKD (72.68%) and TransMIL (48.00%).
  • Reduced Forgetting: The "forgetting" metric (the drop in performance on old classes) was only 13.12 percentage points, compared to over 20 points for most competing MIL frameworks.
  • Efficiency: The model is highly scalable. Adding a new class adds only about $Md+1$ parameters (where $d$ is the feature dimension), meaning the predictor remains compact even as the number of recognized diseases grows.

Limitations and Future Work

While EviSlide-MIL is a significant step forward, the authors note several areas for improvement:

  • Frozen Encoders: The model relies on a frozen feature extractor. If the initial foundation model hasn't seen a specific rare tissue type, EviSlide-MIL cannot "fine-tune" the encoder to see it better.
  • Replay Privacy: Even though only features (not raw images) are stored for replay, this still raises potential data privacy concerns in clinical settings.
  • Scale: The study was limited to 8 subtypes. Real-world pathology involves hundreds of potential diagnoses, which will test the limits of the shared prediction module.

Future work will likely explore "task-free" learning streams and privacy-aware replay mechanisms to make these models even more robust for clinical deployment.

Summary updated

Abstract

Histopathological analysis of whole-slide images (WSIs) plays a critical role in cancer diagnosis, and multiple instance learning (MIL) is widely adopted for WSI analysis under weak supervision. As new tumor subtypes are introduced, extending existing MIL models requires learning new classes while retaining knowledge of previously learned ones. This presents a class-incremental learning challenge, where models must acquire new classes with limited access to historical training data and recognize all learned classes without task identifiers at inference. A major obstacle is forgetting, as updates to feature aggregation and prediction modules can degrade performance on previously learned classes. To address this challenge, we propose \textbf{EviSlide-MIL}, a class-incremental WSI framework that integrates multiple learnable queries per class with a shared prediction module. The queries aggregate frozen patch features into class-specific representations, while the shared prediction module limits parameter growth across incremental stages. By freezing queries associated with previously learned classes, EviSlide-MIL preserves their attention patterns for identical inputs, while experience replay and knowledge distillation further mitigate forgetting as shared parameters are updated. Experiments on eight tumor subtypes from The Cancer Genome Atlas demonstrate that EviSlide-MIL achieves improved performance and reduced forgetting compared with competing methods across most learning settings, demonstrating its potential for scalable and knowledge-preserving WSI classification. Code and pretrained models will be released.
BibTeX
@misc{qian2026evislide,
  author = {Zhouhan Qian and Sicheng Chen and Chad Wong and Tianyi Zhang and Enhui Chai and Zeyu Liu and Fei Xia},
  title = {{EviSlide-MIL: Multi-Query Pooling and Shared Prediction for Class-Incremental Whole-Slide Image Classification}},
  year = {2026},
  eprint = {2611.23212},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23212},
}
Figure