Abstract
Conventional data augmentation realized by performing simple pre-processing operations (e.g., rotation, crop, etc.) has been validated for its advantage in enhancing the performance for medical image segmentation. However, the data generated by these conventional augmentation methods are random and sometimes harmful to the subsequent segmentation. In this paper, we developed a novel automatic learning-based data augmentation method for medical image segmentation which models the augmentation task as a trial-and-error procedure using deep reinforcement learning (DRL). In our method, we innovatively combine the data augmentation module and the subsequent segmentation module in an end-to-end training manner with a consistent loss. Specifically, the best sequential combination of different basic operations is automatically learned by directly maximizing the performance improvement (i.e., Dice ratio) on the available validation set. We extensively evaluated our method on CT kidney tumor segmentation which validated the promising results of our method.
Introduction
Deep learning models for medical image segmentation are notoriously data-hungry. However, obtaining large-scale medical datasets is difficult due to privacy concerns and the intensive manual labor required from medical experts to annotate images. To compensate, researchers typically use data augmentation—artificially expanding the dataset by rotating, flipping, or cropping images. The problem is that these "conventional" methods are often applied randomly or based on human intuition. If an augmentation is too aggressive or irrelevant, it can actually harm the model's performance. Furthermore, in most existing pipelines, the augmentation module and the segmentation module are decoupled; the segmentation model has no way to provide feedback to the augmentation process to say, "This specific transformation helped me learn better."
Methods: Augmentation as a Sequential Decision Process
The authors bridge this gap by modeling data augmentation as a Markov Decision Process (MDP) and solving it using a Dueling Deep Q-Network (DQN).
- The Agent and Environment: The DRL agent (the "Augmentor") interacts with an environment (the segmentation task).
- State Representation: Instead of using raw pixels, the state is defined as high-level features extracted from the CT image using a pre-trained U-Net.
- Action Space: The agent can choose from 12 basic operations: horizontal flipping, rotation, cropping (four directions), warping, zooming, adding noise, increasing/decreasing brightness, and a "terminal" action to end the sequence.
- The Reward Signal: The reward is based on the Dice ratio improvement on a validation set. If a sequence of augmentations leads to a better segmentation result, the agent receives a positive reward.
- End-to-End Training: The augmentation policy and the segmentation network are trained in a joint-learning scheme. The agent learns which sequence of operations works best for specific images, while the segmentation module becomes more robust by training on these optimized samples.
Experimental Setup
The study utilized a specific CT kidney tumor dataset collected from a collaborating hospital:
- Size: 68 patients, totaling 3,753 CT slices.
- Characteristics: Each slice contains one tumor. Images are 512 x 512 pixels with a 1 x 1 mm² resolution.
- Split: 50 cases for training, 5 for validation (to calculate rewards), and 13 for testing.
Key Insights
- Augmentation as a Trajectory: Unlike standard approaches that apply a single random transform, this method views augmentation as a trajectory. A specific combination (e.g., Flip → Crop → Brightness) might be more effective than any single operation.
- Task-Specific Policies: The DRL agent learns that certain operations are more valuable for kidney tumors. For instance, the authors noted that Horizontal Flipping (HF) and Cropping were frequently selected, as they effectively increase the variety of foreground tumor appearances.
- Closing the Feedback Loop: By using the segmentation model's performance as the reward, the augmentation strategy is no longer "blind." It evolves to specifically address the weaknesses of the current segmentation model.
Results and Discussion
The proposed method was compared against "No Augmentation," "Traditional Augmentation," and several state-of-the-art methods, including GAN-based approaches.
- Segmentation Accuracy: The method achieved a Dice Similarity Coefficient (DSC) of 84.0%, outperforming traditional augmentation (83.2%) and significantly beating the baseline without augmentation (74.9%).
- Boundary Precision: The most dramatic improvements were seen in distance-based metrics. The Hausdorff Distance (HD)—a measure of boundary error—dropped to 8.52 mm, compared to 13.00 mm for traditional augmentation and 18.00 mm for no augmentation.
- Visual Quality: Qualitative results showed that the DRL-augmented model was much better at localizing tumor boundaries, which is critical for clinical applications like surgical planning.
Limitations and Future Work
While the results are promising, the authors acknowledge a few areas for growth. The current study focuses strictly on kidney tumors in CT scans; future work is needed to see if these learned policies transfer well to other organs or modalities like MRI. Additionally, combining this DRL approach with Generative Adversarial Networks (GANs) could be a powerful "best of both worlds" scenario—using GANs to generate synthetic data and DRL to find the optimal way to transform it. While training a DRL agent alongside a segmentation network is computationally more expensive than simple random flipping, this is primarily a training-time cost rather than an inference-time one.
Summary updated