Abstract
Deep learning models in medical imaging often struggle with catastrophic forgetting when encountering new data distributions. This paper introduces Atlas Replay, a continual learning framework that replaces raw data rehearsal with privacy-preserving anatomical prototypes. By framing segmentation as an atlas-based registration task, the method ensures anatomical consistency and robust knowledge retention across diverse prostate MRI datasets.
Introduction: The Challenge of Catastrophic Forgetting
In clinical settings, deep learning models are frequently updated as new data arrives from different hospitals or scanners. However, standard models suffer from catastrophic forgetting, where learning new tasks degrades performance on previously mastered ones. While "rehearsal" (storing old data) is a common solution, strict privacy regulations like GDPR often prohibit the long-term storage of raw patient scans.
Furthermore, conventional end-to-end segmentation models (e.g., U-Nets) lack anatomical constraints. When faced with domain shift, they may produce semantically nonsensical results. There is a critical need for a system that balances "plasticity" (learning new domains) with "rigidity" (preserving anatomical knowledge) while respecting patient privacy.
Methods: Atlas Replay and Privacy-Preserving Prototypes
The authors propose Atlas Replay, which revitalizes atlas-based segmentation using modern deep learning.
1. Prototype Building
Instead of storing individual patient scans, the framework creates "prototypes." A small subset of scans ($r=7$) from a specific domain is rigidly aligned and averaged. This process obscures unique patient identifiers while retaining the general structural characteristics of the domain (e.g., specific MRI coil types).
2. Registration-Based Segmentation
During training, the model learns to register (warp) a prototype's pre-defined mask onto a new patient's scan using the VoxelMorph framework. This approach leverages structural domain knowledge as a buffer against forgetting. As new datasets arrive, the model is trained to register the most appropriate prototype, ensuring that the predicted segmentations always maintain a valid anatomical shape.
Experimental Setup and Datasets
The framework was evaluated on seven diverse prostate segmentation datasets: RUNMC, BMC, HCRUDB, UCL, BIDMC, HK, and DecathProst. These datasets encompass various vendors (Siemens, Philips, GE) and acquisition parameters, providing a rigorous test for domain shift and continual learning performance.
Key Results
- Knowledge Retention: Atlas Replay achieved a Backward Transfer (BWT) of -8.13%, significantly outperforming sequential training (-26.60%) and Elastic Weight Consolidation (EWC, -27.48%).
- Generalization: The method outperformed standard U-Net models by an average of 15% in Dice score on unseen domains, demonstrating superior robustness to clinical site variations.
- Privacy Validation: In a user study, senior radiologists could only correctly match a prototype to its source scan with 47.5% precision (near random chance in a 1-of-3 setup), validating the privacy-preserving nature of the averaged prototypes.
Conclusion and Future Directions
Atlas Replay offers a promising path for deploying continual learning in privacy-sensitive medical environments. By shifting from pixel-wise prediction to registration-based transformation, the model maintains high semantic coherence. Future work will focus on automating prototype selection and exploring more rigorous privacy frameworks like Differential Privacy to further enhance clinical utility.
Summary updated