Abstract
Domain shift, caused by variations in MRI scanners and protocols, significantly hinders the generalizability of medical imaging models. This paper presents SpotTUnet, an adaptive architecture that automatically determines which U-Net layers require fine-tuning during supervised domain adaptation. By learning an image-specific policy, SpotTUnet achieves state-of-the-art performance across diverse scanner domains while providing a visual "map" of network sensitivity to domain shift.
Introduction
The deployment of deep learning models in clinical settings is often stalled by domain shift. When a model trained at one site is used at another, differences in hardware and imaging parameters lead to performance drops. Supervised domain adaptation (sDA) is the standard remedy, but selecting the optimal layers for fine-tuning remains an open question. SpotTUnet eliminates this manual search by employing a policy network that learns to select between frozen source weights and fine-tuned target weights on a per-block basis.
Methods
SpotTUnet utilizes a "twin" architecture for each block in a standard U-Net. One path contains frozen weights from the source domain, while the other is trainable. A lightweight ResNet-based policy network generates a binary decision for each block using the Gumbel-Softmax trick, allowing for end-to-end training via backpropagation.
To handle scenarios with limited target data, the authors incorporate $L_1$ regularization. This encourages the model to prioritize the reuse of frozen source layers, mitigating the risk of overfitting when only a few labeled slices are available.
Results
Evaluations were conducted on the Calgary-Campinas-359 (CC359) dataset, focusing on brain segmentation across six distinct scanner domains (Siemens, GE, Philips at 1.5T and 3T).
Performance and Stability
SpotTUnet consistently matched or exceeded the performance of the best manual layer-selection heuristics across the entire spectrum of data availability. Notably, while manual strategies often vary in effectiveness across different scanner pairs, SpotTUnet remained stable across all 30 tested source-target combinations. Interestingly, traditional intensity-based preprocessing like histogram matching was found to be less effective than this neural adaptation approach.
Anatomy of the Shift
The learned policies provide a unique window into how domain shift affects U-Net layers. By visualizing which layers are most frequently selected for fine-tuning, researchers can identify the "anatomy" of the shift's impact.
Analysis of these maps reveals that the encoder is generally more susceptible to domain shift than the decoder in MRI tasks. As the amount of available target data increases, the policy network progressively "unlocks" more layers for fine-tuning, beginning with the encoder. This hierarchical adaptation confirms that low-level feature extraction is highly sensitive to the variations in MRI acquisition hardware.
Discussion and Conclusion
SpotTUnet offers a robust and interpretable solution for domain adaptation in medical imaging. By automating layer selection, it achieves high performance with minimal manual intervention. The discovery of encoder-specific vulnerability suggests that future unsupervised methods should focus on aligning these specific layers to improve cross-site generalizability without requiring additional labels.
Summary updated