TL;DR
This paper introduces a "lifelong learning" approach for brain MRI segmentation that allows a single model to adapt to new scanners and protocols with minimal data, all while retaining performance on previously learned domains. This is achieved by sharing convolutional filters across domains and using domain-specific batch normalization layers, effectively bridging the gap between different data acquisition settings.
What Problem Does This Paper Address?
Medical imaging, particularly MRI, is plagued by variability. Images acquired from different scanners or using slightly altered protocols can have significantly different intensity distributions and contrast, even for the same anatomical structures. This "domain shift" severely degrades the performance of deep learning models, like Convolutional Neural Networks (CNNs), that are trained on data from one source when applied to data from another. This limits the practical utility of these powerful tools in real-world clinical settings where data heterogeneity is the norm. The authors aim to develop a segmentation model that can learn from and adapt to these diverse data sources without requiring extensive retraining or losing knowledge of previously encountered domains.
Methods
The core of the proposed solution lies in a novel application of lifelong multi-domain learning using a single CNN architecture. Instead of training separate models for each scanner or protocol, they propose a unified approach:
- Shared Convolutional Filters: The main convolutional layers of the CNN, responsible for extracting general image features, are shared across all domains. This leverages the underlying similarities in brain anatomy that are present regardless of the acquisition parameters.
- Domain-Specific Batch Normalization (BN) Layers: Batch Normalization is a technique that helps stabilize and accelerate CNN training by normalizing layer inputs. The authors adapt this by making the BN layers domain-specific. This means each domain (scanner/protocol) gets its own set of BN parameters ($\gamma$ and $\beta$). These parameters are responsible for learning the specific intensity shifts and contrast differences between domains, effectively "normalizing" the features for each specific domain.
- Lifelong Learning Adaptation: When a new domain (scanner/protocol) is encountered, the model is adapted by initializing new domain-specific BN parameters based on the closest existing domain and then fine-tuning them with a small number of labeled images (around 4). Crucially, the shared convolutional filters remain fixed during this adaptation, and the BN parameters for older domains are preserved, preventing catastrophic forgetting.
Datasets
The experiments were conducted using four publicly available brain MRI datasets:
- Human Connectome Project (HCP)
- Alzheimers Disease Neuroimaging Initiative (ADNI)
- Autism Brain Imaging Data Exchange (ABIDE)
- Information eXtraction from Images (IXI)
These datasets were split into five domains (D1-D5) representing different scanners and MR modalities (T1w, T2w). Domains D1, D2, and D3 were treated as the initial "known" domains, while D4 and D5 were introduced as "new" domains to test the lifelong learning capability. Pseudo ground truth segmentations were generated using FreeSurfer.
Key Insights and Ideas
- Domain-Specific Normalization is Key: The paper strongly suggests that while the fundamental feature extraction (convolutional filters) can be shared, the specific intensity characteristics of different domains are best handled by domain-specific normalization layers. This is a more nuanced approach than global intensity standardization.
- Lifelong Learning is Achievable for Medical Imaging: The work demonstrates that the principles of lifelong learning, where models continuously learn new tasks without forgetting old ones, can be effectively applied to the challenging domain of medical image segmentation.
- Minimal Data for New Domains: The ability to adapt to new domains with as few as $\approx$ 4 labeled images is a significant practical advantage, reducing the burden of data annotation.
- Preserving Old Knowledge: The critical aspect of lifelong learning is retaining performance on previously learned domains. The proposed method successfully achieves this by keeping the shared filters fixed and only adapting the domain-specific BN layers.
Key Results
- The proposed lifelong learning network (N123) successfully preserves the performance of individual networks trained on each domain (N1, N2, N3) for the initial domains.
- For new domains (D4, D5), the lifelong learning approach significantly closes the performance gap compared to training a dedicated network from scratch (Na) with limited data. The Dice scores for segmentation of various brain structures are shown to be comparable to the dedicated network baseline.
- Qualitative results (visualizations in Figure 2) show that the adapted lifelong learning model produces segmentations on new domains that are visually similar to those from a dedicated network and the ground truth.
- The domain-specific BN modules can be updated for a new domain in approximately 1 hour, a substantial improvement over retraining a full CNN.
Limitations and Future Work
- Pseudo Ground Truth: The authors acknowledge that using FreeSurfer for pseudo ground truth annotations, while a reasonable proxy, is not as ideal as expert-annotated data.
- Future Work: The authors express interest in extending their approach to handle MR modalities that were not present during the initial training. They also highlight the broader implications of their work for addressing data distribution shifts, which are ubiquitous in clinical data.
Summary updated