Training Multi-organ Segmentation Networks with Sample Selection by Relaxed Upper Confident Bound

Yan Wang, Yuyin Zhou, Peng Tang, Wei Shen, Elliot K. Fishman, Alan L. Yuille

TL;DR

This paper introduces a novel sample selection strategy called Relaxed Upper Confident Bound (RUCB) for training medical image segmentation networks. RUCB intelligently selects challenging training samples, mitigating the negative impact of annotation errors, and significantly improves segmentation performance compared to existing methods.

What Problem Does This Paper Address?

Deep convolutional neural networks (CNNs), particularly fully convolutional networks (FCNs), have revolutionized medical image segmentation, enabling automatic identification of organs in scans like CT. While architectural advancements have been a major focus, the paper highlights a less explored but crucial aspect: how to effectively select training samples from a fixed dataset to maximize learning. Current methods often randomly sample slices, but this can be inefficient. More advanced techniques like hard sample mining exist, but they can be susceptible to annotation errors in medical data. Medical images, unlike natural images, often have subtle annotation mistakes (e.g., partially missing organs or incorrect boundaries) that can lead to very high training losses. These "erroneous hard samples" can mislead the network during training, hindering performance. This paper aims to address this by developing a sample selection strategy that can identify and leverage genuinely hard samples while being robust to annotation errors.

Methods

The core contribution is the Relaxed Upper Confident Bound (RUCB) policy for sample selection. This method builds upon the Upper Confident Bound (UCB), a well-established approach for balancing exploration (trying new things) and exploitation (using what's known to be good) in reinforcement learning, specifically in the context of the multi-armed bandit problem.

Here's how it works:

  • UCB Basics: In the context of training, each training slice is treated as an "arm." The "reward" for selecting a slice is its training loss. UCB aims to maximize the cumulative reward by selecting slices that have a high estimated reward (exploitation) and slices that haven't been tried much (exploration). The UCB score for a slice i at iteration t is calculated as: q_i(t) = J_i(t) + sqrt(2 * ln(n) / n_i), where J_i(t) is the average reward (loss) of slice i, n is the total number of iterations, and n_i is the number of times slice i has been selected.
  • The Problem with UCB: The paper argues that UCB can get "stuck" on a small set of very hard samples. In medical imaging, these "very hard" samples might actually be the ones with annotation errors, leading the network to learn incorrect patterns.
  • RUCB's Innovation: RUCB introduces a "relaxation" to the UCB policy. Instead of strictly picking the slice with the highest UCB score, RUCB:
  • Dynamic Hard Sample Range: It defines a dynamic range of "hard" samples based on the distribution of UCB scores (specifically, scores above μ + a * σ, where μ is the mean and σ is the standard deviation of scores).
  • Random Selection within Range: It then randomly selects a slice from this dynamic range of hard samples. This introduces more variance and prevents the algorithm from being overly fixated on a few potentially erroneous samples.
  • Mitigating Annotation Errors: By exploiting a range of hard samples rather than a single "hardest" one, RUCB is less susceptible to the influence of annotation errors, as the impact of any single erroneous sample is diluted.

The paper also describes a bootstrapping procedure where an initial FCN is trained with random sampling, and then RUCB is applied for several "bootstrapping phases" to refine the training.

Datasets

  • Dataset: The study uses a dataset of 120 abdominal CT scans from normal cases.
  • Details: These scans are contrast-enhanced, acquired using Siemens SOMATOM Sensation64 and Definition CT scanners. They contain between 319-1051 slices per scan, with voxel resolutions varying.
  • Segmentation: Sixteen organs were segmented by four radiologists and confirmed by an expert. The dataset is described as high-quality but acknowledges that a small portion of errors is inevitable.
  • Splitting: A standard four-fold cross-validation strategy was employed, with the dataset randomly partitioned into four folds, each containing 30 CT scans. Experiments involved training on three folds and testing on the remaining one, repeated for all combinations.

Key Insights and Ideas

  • Annotation Errors are a Major Hurdle: The paper strongly emphasizes that annotation errors in medical images are a significant problem for training segmentation models, often manifesting as very high training losses that can be mistaken for genuine hard samples.
  • Sample Selection is Crucial: Beyond network architecture, how training data is sampled is a critical factor for achieving high performance in medical image segmentation.
  • UCB's Limitation: While UCB is good for exploration-exploitation, its tendency to focus on the absolute hardest samples can be detrimental when those samples are noisy or incorrectly annotated.
  • RUCB's "Softening" Effect: Relaxing the selection to a range of hard samples, rather than a single extreme, is key to making the training process more robust to annotation noise. This "softening" allows the model to learn from a broader set of challenging cases without being overly influenced by outliers.
  • Balancing Exploration and Exploitation: RUCB provides a more nuanced approach to balancing exploration and exploitation, specifically tailored for the challenges of medical image datasets.

Key Results

  • Significant Performance Boost: The proposed RUCB policy significantly boosts segmentation performance across multiple organs.
  • Improved Average DSC: The average Dice-Sørensen similarity coefficient (DSC) for sixteen segmented organs was improved by RUCB compared to Uniform sampling, Online Hard Example Mining (OHEM), and standard UCB.
  • Specific Organ Improvements: Notable improvements were seen in organs like Adrenal gland (from 29.33% to 36.76%), Celiac AA (34.49% to 38.45%), Duodenum (63.39% to 64.86%), Right kidney (94.48% to 95.40%), Pancreas (77.86% to 78.48%), and SMA (45.36% to 49.59%).
  • RUCB Outperforms UCB: RUCB achieved better results than UCB, even when UCB was trained for more iterations (80K vs. RUCB's 60K total iterations across bootstrapping phases).
  • OHEM Underperforms: Interestingly, OHEM performed worse than Uniform sampling, suggesting that simply mining the absolute highest loss samples without considering potential errors is not effective.
  • Qualitative Results: Visualizations show that UCB tends to select samples with more obvious annotation errors (e.g., incorrect colon segmentation), while RUCB selects samples that are genuinely hard to segment due to complex organ structures and small sizes.

Limitations and Future Work

  • Hyperparameter Sensitivity: The paper mentions that α is an important hyperparameter for RUCB and explores its range. While the results show that a fixed α leads to performance decreases, a more thorough analysis of hyperparameter tuning and its impact could be a future direction.
  • Fixed Dataset Assumption: The bootstrapping procedure described focuses on selecting samples from a fixed dataset. Future work could explore online learning scenarios where new data is continuously acquired.
  • Generalizability: While tested on abdominal CT scans, further validation on other medical imaging modalities and anatomical regions would be beneficial to confirm the broad applicability of RUCB.
  • Annotation Error Detection: Explicitly detecting and handling annotation errors, rather than indirectly mitigating their impact through sample selection, could be another avenue for future research.

Summary updated

Abstract

Convolutional neural networks (CNNs), especially fully convolutional networks, have been widely applied to automatic medical image segmentation problems, e.g., multi-organ segmentation. Existing CNN-based segmentation methods mainly focus on looking for increasingly powerful network architectures, but pay less attention to data sampling strategies for training networks more effectively. In this paper, we present a simple but effective sample selection method for training multi-organ segmentation networks. Sample selection exhibits an exploitation-exploration strategy, i.e., exploiting hard samples and exploring less frequently visited samples. Based on the fact that very hard samples might have annotation errors, we propose a new sample selection policy, named Relaxed Upper Confident Bound (RUCB). Compared with other sample selection policies, e.g., Upper Confident Bound (UCB), it exploits a range of hard samples rather than being stuck with a small set of very hard ones, which mitigates the influence of annotation errors during training. We apply this new sample selection policy to training a multi-organ segmentation network on a dataset containing 120 abdominal CT scans and show that it boosts segmentation performance significantly.
BibTeX
@inproceedings{wang2018training,
  author = {Yan Wang and Yuyin Zhou and Peng Tang and Wei Shen and Elliot K. Fishman and Alan L. Yuille},
  title = {{Training Multi-organ Segmentation Networks with Sample Selection by Relaxed Upper Confident Bound}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2018},
  doi = {10.1007/978-3-030-00937-3\_50},
}
Figure