Federated-Continual Dynamic Segmentation of Histopathology Guided by Barlow Continuity

Niklas Babendererde, Haozhe Zhu, Moritz Fuchs, Jonathan Stieber, Anirban MukhopadhyayWACV 2025

TL;DR

This paper introduces Dynamic Barlow Continuity (DynBC), a novel method to improve AI model robustness in histopathology image segmentation by jointly addressing Client Drift (spatial data shifts in federated learning) and Catastrophic Forgetting (temporal data shifts in continual learning). DynBC achieves this by using a reference dataset to evaluate model updates for shift invariance, ensuring more stable and reliable AI deployment in dynamic, real-world scenarios.

What Problem Does This Paper Address?

Deploying AI for histopathology segmentation holds immense promise for assisting pathologists, but it faces significant challenges in real-world, dynamic environments. Two major hurdles are:

  1. Client Drift (CD): In Federated Learning (FL), where models are trained across multiple institutions without sharing raw data, differences in scanners, staining techniques, or artifacts between hospitals (spatial shifts) can cause the central model to degrade. This is known as Client Drift.
  2. Catastrophic Forgetting (CF): In Continual Learning (CL), models must adapt to new data over time without access to older datasets (due to privacy or storage constraints). Temporal shifts, such as changes in patient populations or acquisition techniques, can cause the model to forget previously learned information, leading to Catastrophic Forgetting.

While both problems stem from data shifts, existing research typically addresses them independently. This paper argues for a unified approach to tackle both spatial and temporal shifts simultaneously, which is crucial for robust and privacy-aware AI in histopathology.

Methods

The core innovation is the Dynamic Barlow Continuity (DynBC) method. It's designed to guide the training process towards a "shift-invariant" parameter configuration. Here's how it works:

  • Barlow Twins Inspiration: DynBC adapts the Barlow Twins objective, originally used for self-supervised learning to measure embedding changes under data shifts.
  • Model Update Evaluation: Instead of comparing embeddings of a single image, DynBC compares the predictions of two models (the current server model and a potential client update) on a set of augmented images from a public reference dataset.
  • DynBC Metric: A small DynBC value indicates that the model update is shift-invariant, meaning it doesn't cause drastic prediction changes when faced with variations in the input data. This is crucial for both spatial (client updates) and temporal (continual learning) continuity.
  • Dynamic Aggregation: During the aggregation step in FL or the update step in CL, DynBC evaluates each potential model update. If an update shows a significant increase in DynBC (exceeding a dynamic threshold factor, typically 2.0), it's considered to be introducing undesirable shift-invariance and is either ignored (in FL) or rolled back (in CL). This ensures the global model remains robust to spatio-temporal shifts.
  • Reference Dataset: A separate, public histopathology dataset is used for evaluation. This preserves privacy as it doesn't involve sensitive patient data from the training clients.

Datasets

The paper uses the following datasets:

  • Training/Evaluation Datasets:
    • BCSS: A histopathology dataset for breast cancer segmentation.
  • Semicol: A histopathology dataset for colorectal cancer segmentation.
  • Reference Dataset:
    • Camelyon17: Used solely for measuring prediction distances within the DynBC evaluation, not for training or testing the segmentation model itself.

The datasets were split by patients, with specific percentages for training, testing, and validation. Augmentations (Gaussian Blur, Brightness) were applied to simulate data shifts.

Key Insights and Ideas

  • Unified Approach: The most significant insight is that Client Drift and Catastrophic Forgetting, despite arising from different learning paradigms (FL vs. CL), share a common root cause: spatio-temporal distribution shifts. Addressing them jointly is more effective than individual methods.
  • Prediction-Based Continuity: Evaluating model updates based on their prediction continuity (using DynBC) on a reference dataset is a novel way to ensure robustness, rather than relying solely on parameter similarity.
  • Privacy-Preserving Evaluation: Using a public reference dataset for DynBC evaluation is a clever way to maintain privacy while still assessing shift invariance.
  • Dynamic Thresholding: The use of a dynamic threshold factor (relative to the maximum observed distance) makes the method adaptable to different model complexities and scales, unlike fixed absolute thresholds.

Key Results

DynBC demonstrated significant improvements across various scenarios:

  • Client Drift: Achieved dice score improvements from 15.8% to 71.6% in Client Drift scenarios.
  • Catastrophic Forgetting: Showed improvements from 42.5% to 62.8% in Catastrophic Forgetting scenarios.
  • Combined CD/CF: In combined scenarios, DynBC consistently outperformed baselines and individual methods like FedAdam (for CD) and Rehearsal (for CF), highlighting its advantage in tackling both problems concurrently. For instance, in one combined scenario, DynBC achieved a dice score of 0.731±0.118, significantly better than the baseline (0.579±0.062).
  • Ablation Studies: Augmenting the reference dataset improved DynBC's ability to evaluate shift invariance, particularly for Client Drift. A threshold factor of 2.0 was found to provide the best performance.

Limitations and Future Work

  • Reference Dataset Requirement: The primary limitation is the need for a separate reference dataset. While this is not an issue for histopathology due to the availability of public data, it could limit applicability in domains with extremely scarce public data.
  • Threshold Tuning: While the dynamic threshold is an improvement, finding the optimal factor might still require some tuning for specific applications.
  • Future Work: The authors suggest that DynBC's ability to enable robust and privacy-aware deployment of AI in histopathology is a significant step. The method's potential for broader applications beyond histopathology is also implied.

Summary updated

BibTeX
@inproceedings{babendererde2025federated,
  author = {Niklas Babendererde and Haozhe Zhu and Moritz Fuchs and Jonathan Stieber and Anirban Mukhopadhyay},
  title = {{Federated-Continual Dynamic Segmentation of Histopathology Guided by Barlow Continuity}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2025},
}
Figure