Abstract
Automatic segmentation methods based on deep learning have recently demonstrated state-of-the-art performance, outperforming the ordinary methods. Nevertheless, these methods are inapplicable for small datasets, which are very common in medical problems. To this end, we propose a knowledge transfer method between diseases via the Generative Bayesian Prior network. Our approach is compared to a pre-train approach and random initialization and obtains the best results in terms of Dice Similarity Coefficient metric for the small subsets of the Brain Tumor Segmentation 2018 database (BRATS2018).
Introduction
Deep learning has revolutionized medical image segmentation, but it remains heavily dependent on large, expertly labeled datasets. In clinical practice, obtaining such data is often difficult due to the rarity of specific pathologies and the high cost of manual annotation by radiologists. While transfer learning—fine-tuning a model pre-trained on a large source dataset—is a standard solution, it often fails in medical imaging. MRI scans vary significantly across diseases, protocols, and hardware. Standard fine-tuning or freezing layers can lead to "negative transfer," where features from a source task (e.g., Multiple Sclerosis) are incompatible with a target task (e.g., Brain Tumors), trapping the model in poor local minima.
Methods: Deep Weight Prior (DWP)
The authors propose UNet-DWP, a Bayesian approach that utilizes an implicit generative prior to transfer knowledge. The method consists of three stages:
- Source Training: A U-Net is trained on a large source dataset (Multiple Sclerosis Human Brain MR Imaging) to extract high-quality convolutional filters.
- Learning the Prior: A Variational Autoencoder (VAE) is trained on these filters to learn the underlying distribution of "good" kernels.
- Target Training: When training on a small target dataset (BRATS2018), the VAE serves as a prior distribution $p(w)$ within a Variational Inference framework.
This approach regularizes the target model, ensuring its filters remain structured and non-noisy, even when training data is extremely limited. By using a VAE, the authors create an "implicit" prior that captures complex, high-dimensional relationships between weights in a convolutional filter, which is more powerful than simple Gaussian priors.
Results
The UNet-DWP method was evaluated against random initialization (UNet-RI), standard pre-training (UNet-PR), and pre-training with frozen layers (UNet-PRf) on small subsets of the BRATS2018 dataset containing only 5, 10, 15, or 20 images.
As shown in the table above, UNet-DWP achieves superior performance across all sample sizes. With only 5 training images, it reaches a score of 0.52, outperforming both pre-training (0.49) and random initialization (0.50). With 20 training images, UNet-DWP reaches 0.63, significantly outperforming the baseline random initialization at 0.60 and pre-training at 0.58. Notably, standard transfer learning (UNet-PR) often performed worse than starting from scratch (UNet-RI), highlighting how domain shifts in medical imaging can lead to negative transfer when using traditional methods.
Discussion and Conclusion
A key insight from this work is that medical imaging features are highly disease-specific. The failure of frozen layers (UNet-PRf), which performed the worst in all tests, suggests that forcing a model to use rigid features from a different domain is counterproductive. The Deep Weight Prior allows for flexibility while maintaining the structural integrity of the filters.
While effective, the method currently requires identical architectures for source and target tasks to ensure filter dimensions match the VAE's learned space. It also introduces additional computational overhead due to the VAE training and Variational Inference. Future research will explore cross-modality transfer and methods to bridge larger domain gaps in resolution and contrast.
Summary updated