Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search

Dazhou Guo, Dakai Jin, Zhuotun Zhu, Tsung-Ying Ho, Adam P. Harrison, Chun-Hung Chao, Jing Xiao, Alan Yuille, Chien-Yu Lin, Le Lu

TL;DR

The SOARS (Stratified Organ at Risk Segmentation) framework significantly advances automated radiotherapy planning by mimicking clinical workflows. By stratifying 42 different head and neck organs into categories based on difficulty and using Neural Architecture Search (NAS) to tailor network structures for each, it achieves a new state-of-the-art Dice score of 75.14%, outperforming standard monolithic models.

What Problem Does This Paper Address?

In radiotherapy for head and neck (H&N) cancer, radiation oncologists must precisely delineate "Organs at Risk" (OARs) to ensure high-dose radiation hits the tumor while sparing healthy tissue. Manually segmenting these structures is a Herculean task: a single patient may require the delineation of over 40 OARs, taking upwards of two hours and suffering from high inter-practitioner variability.

Automating this with Deep Learning is challenging because H&N OARs are a "mixed bag." They range from large, high-contrast structures (like the mandible) to tiny, low-contrast ones (like the optic chiasm) packed into a dense spatial area. Standard Fully Convolutional Networks (FCNs) often struggle to optimize for all these variations simultaneously, especially as the number of OARs increases.

Methods

The authors introduce SOARS, which tackles complexity through two dimensions of stratification:

  1. Processing Stratification: Inspired by how radiologists use "easy" organs as landmarks to find "hard" ones, SOARS divides OARs into three categories:
  2. Anchor OARs: High-contrast, easy-to-locate structures (e.g., brain stem, mandible). These are segmented first.
  3. Mid-level OARs: Soft-tissue organs with lower contrast (e.g., parotid glands). These are segmented using the predicted Anchor OARs as additional spatial guidance.
  4. Small & Hard (S&H) OARs: Tiny structures with very poor contrast (e.g., cochlea, pineal gland). SOARS uses a "detection-then-segmentation" approach, first regressing a heat map to find the center and then zooming in for fine-grained segmentation, again guided by Anchor OAR predictions.
  5. Architectural Stratification: Instead of using a one-size-fits-all backbone, the authors use Differentiable Neural Architecture Search (NAS). For each OAR category, the system automatically searches for the optimal combination of 2D, 3D, or Pseudo-3D (P3D) convolutions and kernel sizes (3x3x3 vs 5x5x5).
Figure 2. (a) SOARS stratifies OAR segmentation across two dimensions: distinct processing frameworks and distinct architectures. We execute the latter using differentiable NAS. (b) depicts illustrates the backbone network (P-HNN) with NAS, which allows for an automatic selection across 2D, 3D, P3D convolutions. (c) demonstrates the NAS search space setting.

The underlying backbone for all branches is the P-HNN (Progressive Holistically-Nested Network), which excels at propagating features across different scales.

Datasets

  • H&N 42 Dataset: A comprehensive, newly introduced dataset consisting of 142 H&N cancer patients, each with 42 manually labeled OARs. This represents the most complete OAR dataset reported to date.
  • MICCAI 2015 H&N Challenge: A public benchmark dataset (33 training, 15 test cases) used to evaluate the generalizability of the model against existing state-of-the-art methods.

Key Insights and Ideas

  • Hierarchical Guidance Matters: Using easy-to-segment "Anchors" to provide context for harder organs mimics human expertise and significantly reduces false positives in soft-tissue areas.
  • 3D Isn't Always King: A surprising finding from the NAS process was that 3D kernels are not always the best choice. For Anchor and Mid-level OARs, the search often favored mixed 2D and P3D kernels, which save memory and computation while maintaining performance. However, for S&H OARs, 3D kernels were preferred, likely because these tiny objects rely heavily on volumetric spatial context to be distinguished from noise.
  • Stratification Eases Optimization: By decoupling the 42-organ task into smaller, specialized sub-tasks, the model avoids the "gradient interference" that happens when a single network tries to learn vastly different features simultaneously.

Key Results

SOARS demonstrated substantial improvements over previous state-of-the-art (SOTA) methods like UaNet and standard 3D U-Nets.

  • Quantitative Performance: On the H&N 42 dataset, SOARS achieved an average Dice Similarity Coefficient (DSC) of 75.14%, a significant jump from the 70.44% achieved by UaNet. It also reduced the Hausdorff Distance (HD)—a measure of boundary error—from 9.20mm to 6.98mm.
  • S&H Breakthrough: The most dramatic gains were seen in the "Small & Hard" category, where DSC improved from ~62% (baseline) to nearly 72%.
  • Generalizability: On the public MICCAI 2015 challenge, SOARS achieved a DSC of 82.4%, setting a new benchmark and outperforming the previous best result by 1.2%.
Table 3. Quantitative results of different approaches on segmenting the 42 H&N OARs using the 4-fold cross validation. Our proposed SOARS achieves the best performance in all metrics (indicated in bold).

Qualitatively, the impact of NAS and anchor guidance is clear, particularly in resolving the boundaries of soft-tissue organs that standard models often miss or over-segment.

Figure 3. Qualitative mid-level OAR segmentation using different setups. The seven columns are seven representative axial slices in the RTCT image. For better comparison, we use red arrows to indicate the improvements. The 1st row is the RTCT image with OAR delineations of a radiation oncologist. The 3rd row shows the impact of using anchor OARs, which can help the segmentation of soft-tissue mid-level OARs. The 4th demonstrates the impact of NAS, indicating the necessity of adapting network architectures for different OARs.

Limitations and Future Work

While highly effective, the current NAS approach is applied at the category level rather than the individual organ level to keep computation manageable. Searching for a unique architecture for every single one of the 42 OARs is currently "computationally unaffordable."

Future work could explore more efficient NAS methods to allow for even finer-grained architectural tailoring. Additionally, while the "Anchor, Mid, S&H" split is clinically grounded, exploring automated ways to discover these optimal groupings could further refine the pipeline.

Summary updated

Abstract

OAR segmentation is a critical step in radiotherapy of head and neck (H&N) cancer, where inconsistencies across radiation oncologists and prohibitive labor costs motivate automated approaches. However, leading methods using standard fully convolutional network workflows that are challenged when the number of OARs becomes large, e.g. > 40. For such scenarios, insights can be gained from the stratification approaches seen in manual clinical OAR delineation. This is the goal of our work, where we introduce stratified organ at risk segmentation (SOARS), an approach that stratifies OARs into anchor, mid-level, and small & hard (S&H) categories. SOARS stratifies across two dimensions. The first dimension is that distinct processing pipelines are used for each OAR category. In particular, inspired by clinical practices, anchor OARs are used to guide the mid-level and S&H categories. The second dimension is that distinct network architectures are used to manage the significant contrast, size, and anatomy variations between different OARs. We use differentiable neural architecture search (NAS), allowing the network to choose among 2D, 3D or Pseudo-3D convolutions. Extensive 4-fold cross-validation on 142 H&N cancer patients with 42 manually labeled OARs, the most comprehensive OAR dataset to date, demonstrates that both pipeline- and NAS-stratification significantly improves quantitative performance over the state-of-the-art (from 69.52% to 73.68% in absolute Dice scores). Thus, SOARS provides a powerful and principled means to manage the highly complex segmentation space of OARs.
BibTeX
@inproceedings{guo2020organ,
  author = {Dazhou Guo and Dakai Jin and Zhuotun Zhu and Tsung-Ying Ho and Adam P. Harrison and Chun-Hung Chao and Jing Xiao and Alan Yuille and Chien-Yu Lin and Le Lu},
  title = {{Organ at Risk Segmentation for Head and Neck Cancer using Stratified Learning and Neural Architecture Search}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2020},
  doi = {10.1109/cvpr42600.2020.00428},
}
Figure