Abstract
Continual learning (CL) methods designed for natural image classification often fail to reach basic quality standards for medical image segmentation. Atlas-based segmentation, a well-established approach in medical imaging, incorporates domain knowledge on the region of interest, leading to semantically coherent predictions. This is especially promising for CL, as it allows us to leverage structural information and strike an optimal balance between model rigidity and plasticity over time. When combined with privacy-preserving prototypes, this process offers the advantages of rehearsal-based CL without compromising patient privacy. We propose Atlas Replay, an atlas-based segmentation approach that uses prototypes to generate high-quality segmentation masks through image registration that maintain consistency even as the training distribution changes. We explore how our proposed method performs compared to state-of-the-art CL methods in terms of knowledge transferability across seven publicly available prostate segmentation datasets. Our results show that Atlas Replay is both robust and generalizes well to yet-unseen domains while being able to maintain knowledge, unlike end-to-end segmentation methods.
Introduction
In medical imaging, AI models frequently encounter data drift. A model trained on scans from one hospital may fail when deployed at another due to differences in scanner vendors, imaging protocols, or patient demographics. While Continual Learning (CL) aims to help models learn from new data sequentially without forgetting previous knowledge, standard methods struggle with medical segmentation for two primary reasons: semantic incoherence (producing anatomically impossible masks) and strict privacy constraints (like GDPR) that prevent the long-term storage of sensitive patient scans required for traditional "rehearsal" methods.
Researchers have developed Atlas Replay, a novel framework that swaps traditional end-to-end segmentation for an atlas-based approach. By using privacy-preserving anatomical templates and deep-learning-based image registration, the method effectively eliminates "catastrophic forgetting" while maintaining high clinical standards.
Methods: The Atlas Replay Framework
Atlas Replay moves away from predicting masks pixel-by-pixel. Instead, it utilizes Atlas-based segmentation through three core components:
- Prototypes: The system creates anatomical templates generated by averaging multiple patient scans. These prototypes capture essential geometry while obscuring individual patient identities.
- VoxelMorph Registration: The model uses the VoxelMorph framework (a specialized U-Net) to learn how to "warp" or register a prototype's known segmentation mask onto a new patient scan.
- Pseudo-Rehearsal: During training, the model "replays" these anonymized prototypes instead of raw patient data. This provides the benefits of rehearsal-based learning without compromising privacy.
Key Insights: Balancing Rigidity and Plasticity
A central challenge in CL is the rigidity vs. plasticity trade-off: the model must be plastic enough to learn new tasks but rigid enough to remember old ones. Standard models often produce "noisy" segmentations when trying to find this balance.
Atlas Replay addresses this by using geometry as a prior. Unlike natural images, organs have stable geometric properties. Atlas-based methods leverage this domain knowledge to ensure segmentations remain semantically coherent. Furthermore, by building prototypes from averaged scans, the authors created a representation that senior radiologists confirmed makes it nearly impossible to identify original patients.
Results and Performance
The study utilized seven publicly available prostate MRI datasets (RUNMC, BMC, HCRUDB, UCL, BIDMC, HK, and DecathProst) representing diverse scanner vendors (Siemens, Philips, GE) and acquisition parameters.
- Superior Knowledge Retention: Atlas Replay outperformed state-of-the-art CL methods like EWC, RWalk, and ILT, achieving a superior balance between Backward Transfer (BWT) and Forward Transfer (FWT).
- Stability: While end-to-end models often suffered from catastrophic forgetting on earlier datasets, Atlas Replay maintained consistent performance across all seven tasks.
- Quantitative Edge: In direct comparisons of generalizability, the registration-based approach maintained a mean Dice score roughly 15% higher than standard U-Net models when tested across different clinical sites.
Conclusion and Future Directions
Atlas Replay demonstrates that incorporating anatomical priors through registration can solve the most pressing issues in medical continual learning. While the current method requires manual prototype selection based on scan type (e.g., use of endorectal coils), future work may involve "self-growing" atlases that automatically adjust to increasingly diverse anatomy. Additionally, exploring more mathematically rigorous privacy guarantees, such as Differential Privacy, could further strengthen the framework's clinical utility.
Summary updated