Abstract
Automated prostate segmentation in MRI is highly demanded for computer-assisted diagnosis. Recently, a variety of deep learning methods have achieved remarkable progress in this task, usually relying on large amounts of training data. Due to the nature of scarcity for medical images, it is important to effectively aggregate data from multiple sites for robust model training, to alleviate the insufficiency of single-site samples. However, the prostate MRIs from different sites present heterogeneity due to the differences in scanners and imaging protocols, raising challenges for effective ways of aggregating multi-site data for network training. In this paper, we propose a novel multi-site network (MS-Net) for improving prostate segmentation by learning robust representations, leveraging multiple sources of data. To compensate for the inter-site heterogeneity of different MRI datasets, we develop Domain-Specific Batch Normalization layers in the network backbone, enabling the network to estimate statistics and perform feature normalization for each site separately. Considering the difficulty of capturing the shared knowledge from multiple datasets, a novel learning paradigm, i.e., Multi-site-guided Knowledge Transfer, is proposed to enhance the kernels to extract more generic representations from multi-site data. Extensive experiments on three heterogeneous prostate MRI datasets demonstrate that our MS-Net improves the performance across all datasets consistently, and outperforms state-of-the-art methods for multi-site learning.
The Challenge of Multi-Site Heterogeneity
Training accurate Convolutional Neural Networks (CNNs) for medical image segmentation requires large amounts of annotated data, which is notoriously scarce at any single hospital or clinical site. A logical solution is to pool data from multiple sites. However, multi-site MRI data is highly heterogeneous. Different hospitals use different scanner manufacturers (e.g., Siemens vs. Philips), magnetic field strengths, coil types, and imaging protocols. This creates a "domain shift" where a model trained on one site's data performs poorly on another.
Previous studies have shown that directly mixing this heterogeneous data to train a single network (a "Joint" approach) often yields marginal improvements or even performs worse on local data than a model trained exclusively on that local data (a "Separate" approach). The authors tackle the challenge of effectively aggregating multi-site data to boost performance across all participating sites without falling victim to inter-site heterogeneity.
Methodology: The MS-Net Framework
The authors propose the Multi-Site Network (MS-Net), which relies on two core innovations to disentangle site-specific noise from shared anatomical features:
1. Domain-Specific Batch Normalization (DSBN)
Instead of sharing Batch Normalization (BN) layers across all data, the network's backbone assigns a dedicated BN layer to each specific site. Because BN layers track the mean and variance of feature maps, forcing all sites through one BN layer muddles the statistics. DSBN allows the network to normalize features individually for each site, absorbing domain-specific nuances so the shared convolutional filters can focus on generic prostate features.
2. Multi-site-guided Knowledge Transfer (MSKT)
To prevent the shared convolutional kernels from just memorizing site-specific quirks, the authors introduce a novel training paradigm. Alongside the main "universal network," they train several "auxiliary branches" (one for each site) that act as site-specific experts. During training, a knowledge transfer loss forces the universal network to mimic the probability maps of these auxiliary branches. This acts as a powerful regularizer, pushing the universal network to find a robust global minimum that works well across all datasets.
Experimental Results
The authors evaluated their method using T2-weighted prostate MRI scans from three highly heterogeneous sites (Site A, B, and C). MS-Net achieved an overall Dice Coefficient of 91.66% across the three sites, significantly outperforming both the Joint baseline (90.25%) and the Separate baseline (90.56%).
Beyond the Dice score, MS-Net achieved an overall Average Symmetric Distance (ASD) of 0.77mm, compared to 0.87mm for both the Joint and Separate baselines, indicating much tighter boundary segmentation. This improvement is visually evident in the model's ability to handle varying resolutions and contrast levels across different clinical sources.
Conclusion and Future Work
MS-Net demonstrates that by intelligently handling domain-specific statistics through DSBN and leveraging site-specific expertise via MSKT, it is possible to effectively aggregate heterogeneous medical data. While the model currently requires site identification at inference time to select the correct BN layer, it sets a new benchmark for multi-site prostate segmentation. Future work will focus on "Domain Generalization" to ensure these models can perform seamlessly on completely unseen external hospitals without prior site-specific training.
Summary updated