Task-Agnostic Continual Hippocampus Segmentation for Smooth Population Shifts

Camila González, Amin Ranem, Ahmed E. Othman, Anirban Mukhopadhyay

TL;DR

This paper introduces ODEx, a novel task-agnostic continual learning method for medical image segmentation that handles gradual, unlabeled shifts in data distribution. ODEx uses out-of-distribution detection to dynamically expand the model pool, achieving strong performance across tasks without needing explicit task boundaries or labels, a crucial step towards real-world clinical AI.

What Problem Does This Paper Address?

Deep learning models typically assume data is "i.i.d." (independent and identically distributed), meaning training and testing data come from the same distribution. This assumption breaks down in real-world applications like healthcare, where data distributions can change over time due to evolving imaging equipment, patient populations, or disease patterns. Continual learning (CL) aims to address this by allowing models to learn new tasks sequentially without forgetting old ones. However, most CL methods rely on clearly defined task boundaries and explicit task labels, which are often unavailable in dynamic clinical settings. This paper tackles the challenge of "task-agnostic" continual learning, where the model must adapt to distribution shifts without knowing when a shift occurs or which task it's currently facing, mimicking the smooth, unlabeled transitions common in clinical practice.

Methods

The core of ODEx is an "expansion-based" approach combined with continual out-of-distribution (OOD) detection. Instead of a single model, ODEx maintains a pool of models.

  • Training: During training, ODEx monitors features from the early layers of the network (specifically, batch normalization statistics). It uses the Mahalanobis distance to detect deviations from the learned distribution of previous stages. If a significant shift is detected, ODEx expands its model pool by initializing a new model. This new model is then trained on the current data. Crucially, ODEx does not store past data (to respect privacy) and does not require task labels.
  • Inference: For inference, ODEx calculates the Mahalanobis distance of the input data to the distribution represented by each model in its pool. It then selects the model that best fits the current data distribution to perform the segmentation.

This approach aims to balance "rigidity" (preserving knowledge from past tasks) and "plasticity" (learning new tasks) without explicit task supervision.

Datasets

The paper evaluates ODEx on two scenarios of hippocampus segmentation using T1-weighted MRI data:

  1. Shifting Source: This scenario combines data from three public datasets: HarP (healthy and Alzheimer's patients), Dryad (healthy adults), and Decathlon (healthy and schizophrenia patients). The data streams are created by gradually shifting the proportion of cases from each source.
  2. Transformed: This scenario uses the Decathlon dataset and artificially transforms it by applying intensity rescaling (contrast stretching) and affine transformations (scaling, rotation, translation) to simulate gradual distribution shifts.

The authors used the first 80/20 split generated by the nnUNet framework for the initial training data and ensured that test cases remained as such across both scenarios.

Key Insights and Ideas

  • Task-agnosticism is crucial for real-world AI: The paper strongly argues that current CL benchmarks are too simplistic and that methods must handle unlabeled, gradual shifts to be useful in clinical settings.
  • OOD detection as a "shift detector": Using Mahalanobis distance on batch normalization statistics is an effective, lightweight way to signal when a new model is needed, without requiring access to old data.
  • Expansion-based models can be efficient: By managing the model pool and only expanding when necessary, ODEx avoids the linear growth of parameters seen in some other expansion methods, keeping resource usage comparable to sequential learning.
  • Hippocampus segmentation is a challenging benchmark: The sensitivity of hippocampus segmentation to distribution shifts makes it a good testbed for evaluating continual learning methods.

Key Results

  • ODEx significantly outperforms other state-of-the-art continual learning approaches (EWC, MiB, RW, PLOP, LwF) in terms of Dice score for hippocampus segmentation across both the "Shifting Source" and "Transformed" scenarios.
  • While EWC retains old knowledge well, it suffers from poor plasticity (low FWT). Conversely, methods like MiB show good plasticity but forget previous tasks. ODEx achieves a good balance, demonstrating high FWT and maintaining performance on earlier tasks.
  • Resource utilization is comparable to regular sequential learning, with only a marginal increase in training time and no increase in GPU memory. The storage for multiple model parameters is also kept low (less than 0.8 GB for the explored scenarios).
  • Qualitative results show ODEx effectively segments the hippocampus body and tail, maintaining shape integrity, unlike some other methods that struggle with these regions after sequential training.

Limitations and Future Work

  • Model Pool Size: While ODEx keeps the model pool size manageable, it does grow. Future work could explore if it's sufficient to store only a subset of domain-specific parameters (e.g., last decoder blocks or batch normalization layers) rather than entire models.
  • OOD Detection Threshold: The paper mentions setting the OOD detection threshold ($\xi$) based on normalized distances. Further research could explore more adaptive or robust methods for setting this threshold.
  • Generalizability: While tested on hippocampus segmentation, further validation on other medical imaging tasks and different types of distribution shifts would be beneficial.
  • Code Release: The authors have released their code and models, encouraging further research in this area.

Summary updated

Abstract

Most continual learning methods are validated in settings where task boundaries are clearly defined and task identity information is available during training and testing. We explore how such methods perform in a task-agnostic setting that more closely resembles dynamic clinical environments with gradual population shifts. We propose ODEx, a holistic solution that combines out-of-distribution detection with continual learning techniques. Validation on two scenarios of hippocampus segmentation shows that our proposed method reliably maintains performance on earlier tasks without losing plasticity.
BibTeX
@inproceedings{gonzalez2022task,
  author = {Camila González and Amin Ranem and Ahmed E. Othman and Anirban Mukhopadhyay},
  title = {{Task-Agnostic Continual Hippocampus Segmentation for Smooth Population Shifts}},
  booktitle = {DART @ MICCAI},
  year = {2022},
  doi = {10.1007/978-3-031-16852-9\_11},
}
Figure