Abstract
Supervised learning-based segmentation methods typically require a large number of annotated training data to generalize well at test time. In medical applications, curating such datasets is not a favourable option because acquiring a large number of annotated samples from experts is time-consuming and expensive. Consequently, numerous methods have been proposed in the literature for learning with limited annotated examples. Unfortunately, the proposed approaches in the literature have not yet yielded significant gains over random data augmentation for image segmentation, where random augmentations themselves do not yield high accuracy. In this work, we propose a novel task-driven data augmentation method for learning with limited labeled data where the synthetic data generator, is optimized for the segmentation task. The generator of the proposed method models intensity and shape variations using two sets of transformations, as additive intensity transformations and deformation fields. Both transformations are optimized using labeled as well as unlabeled examples in a semi-supervised framework. Our experiments on three medical datasets, namely cardiac, prostate and pancreas, show that the proposed approach significantly outperforms standard augmentation and semi-supervised approaches for image segmentation in the limited annotation setting.
Introduction
Deep learning models for medical image segmentation are notoriously data-hungry. While human experts can generalize from a few examples, neural networks often require hundreds of annotated scans to handle variations in patient anatomy and scanner characteristics. Standard Data Augmentation (DA) techniques, such as random rotations or elastic deformations, are often "blind" to the downstream task; they apply transformations without regard for whether they improve the network's ability to learn segmentation boundaries. This paper introduces a semi-supervised, task-driven data augmentation method that learns to generate spatial and intensity transformations specifically optimized to improve segmentation performance, particularly in low-data regimes.
Methodology
The proposed framework utilizes neural networks to generate augmentations instead of relying on hard-coded mathematical transformations. The system comprises two conditional generators:
- Deformation Field Generator ($G_V$): Learns to output dense, per-pixel spatial deformations to model shape variations.
- Additive Intensity Field Generator ($G_I$): Learns to output additive intensity masks to model texture and contrast variations.
A key innovation is the task-driven nature of these generators: they are trained jointly with the segmentation network ($S$). The segmentation loss is backpropagated into the generators, rewarding them for creating samples that minimize segmentation error. To ensure realism and prevent the generation of useless noise, a semi-supervised approach is employed. A Discriminator ($D$) and a pool of unlabeled images ($x_{UL}$) are used to ensure that augmented images match the statistical distribution of the real patient population. Furthermore, a Large Deviation (LD) loss is introduced to force the generators to produce significant transformations, preventing them from simply outputting the original images.
Experimental Setup
The method was evaluated in limited-data scenarios (using 1 or 3 labeled 3D volumes) across three public datasets:
- Cardiac Dataset: MRI scans from the MICCAI 2017 ACDC challenge.
- Prostate Dataset: T2-weighted MRI scans from the MICCAI 2018 Medical Segmentation Decathlon.
- Pancreas Dataset: CT scans from the MICCAI 2018 Medical Segmentation Decathlon.
Results and Discussion
The proposed method consistently outperformed several baselines, including standard affine transformations, random elastic/intensity changes, MixUp, and self-training.
Key Insights
- Task-Driven Realism: Interestingly, the generated images are not always anatomically realistic to the human eye. However, because they are optimized for the segmentation task, they provide more effective training signals than perfectly realistic GAN outputs.
- Joint Optimization: Training the generator in tandem with the segmentation network yields significantly better results than independent optimization.
- Data Efficiency: The performance gap is most pronounced when labeled data is extremely scarce (1-3 volumes). For instance, on the Cardiac dataset with 1 labeled volume, the joint optimization achieved a Dice Similarity Coefficient (DSC) of 0.834 for the Left Ventricle, compared to 0.819 for standard random augmentations.
Conclusion
This work demonstrates that learning a task-specific augmentation strategy can significantly mitigate the challenges of limited annotated data in medical imaging. While computationally more expensive than standard DA due to joint training and hyperparameter tuning, the gains in segmentation accuracy are substantial. Future work may focus on adapting this approach to handle severe domain shifts between different scanner types.
Summary updated