Alleviating Class Imbalance in Semi-supervised Multi-organ Segmentation via Balanced Subclass Regularization

Zhenghao Feng, Lu Wen, Binyu Yan, Jiaqi Cui, Yan Wang

TL;DR

BSR-Net addresses the chronic "small organ" problem in medical image segmentation by re-imagining class imbalance as a label-grouping challenge. By automatically splitting large organs into multiple "subclasses" to match the pixel counts of smaller organs, the researchers create a balanced auxiliary task that regularizes the main model, significantly improving accuracy for tiny, difficult-to-segment structures in semi-supervised settings.

What Problem Does This Paper Address?

In Multi-organ Segmentation (MoS), size matters. A liver is massive compared to an adrenal gland, leading to a massive "class imbalance" where a neural network can achieve high overall accuracy simply by ignoring small organs and focusing on the big ones.

While Semi-Supervised Learning (SSL) helps reduce the need for expensive manual annotations by using unlabeled data, it often exacerbates this imbalance. Most existing solutions (like re-weighting or over-sampling) are designed for fully supervised scenarios and don't translate well to unlabeled data. The authors identify a critical gap: we need a way to force SSL models to learn "unbiased knowledge" that treats small and large organs with equal importance, even when the data itself is heavily skewed.

Methods

The authors propose BSR-Net (Balanced Subclass Regularization Network), which operates in two distinct phases:

  1. Phase I: Class-balanced Subclass Generation
    • The team first trains a standard U-Net on the limited labeled data available.
  2. They extract pixel-level features and apply a Balanced Clustering algorithm. Unlike standard K-means, this algorithm adjusts the number of clusters based on the original organ size.
  3. The Result: Large organs (like the liver) are subdivided into many subclasses, while small organs (like the adrenal glands) remain as single or few subclasses. This creates a new set of "subclass labels" where every class has a roughly equal number of pixels.
  4. Phase II: Balanced Subclass Regularization
    • Using a Mean Teacher framework (a standard SSL architecture), they implement a multi-task learning strategy.
  5. The network has one shared encoder but two separate "heads" (decoders): one for the original MoS task and an auxiliary head for the Subclass Segmentation (SCS) task.
  6. Task Consistency Loss: This is the core innovation. The model is forced to ensure that its subclass predictions, when mapped back to their original categories, match the main MoS predictions. This "regularizes" the main task with the unbiased, balanced knowledge learned by the subclass head.

Datasets

The model was evaluated on two major public benchmarks:

  • MICCAI FLARE 2022: A subset containing 135 CT volumes with 13 labeled organs.
  • WORD (Whole abdominal Organ Dataset): 150 CT volumes featuring 16 different organ structures.
  • The researchers tested the model under "low-data" regimes, specifically using only 5, 10, or 15 labeled volumes to simulate real-world clinical constraints.

Key Insights and Ideas

  • Structural Rebalancing: Instead of just changing loss weights (which can be unstable), BSR-Net changes the granularity of the task. By making the network distinguish between different parts of a large organ, it naturally spends as much "representational effort" on those parts as it does on a whole small organ.
  • Unlabeled Data as a Bridge: Because the subclass generation happens in the feature space, the "balanced" logic can be applied to unlabeled data during the consistency training phase, overcoming a major limitation of traditional re-weighting schemes.
  • Shared Feature Space: By sharing an encoder between the imbalanced main task and the balanced auxiliary task, the "small organ" features discovered in the subclass task are automatically injected into the main segmentation pipeline.

Key Results

BSR-Net consistently outperformed state-of-the-art (SOTA) semi-supervised methods like URPC and EVIL:

  • Superiority in Low-Data Scenarios: With only 5 labeled samples on the FLARE dataset, BSR-Net outperformed the second-best method (EVIL) by 4.60% in Mean Dice score and 5.97% in Jaccard Index.
  • Small Organ Gains: The most dramatic improvements were seen in tiny structures. For example, on the Right Adrenal Gland (RAG), BSR-Net achieved a Dice score of 66.56% compared to the next best at ~62%.
  • Ablation Proof: The authors showed that using balanced clustering (Phase I) was significantly more effective than using standard k-means, proving that pixel-count parity is the "secret sauce" for subclass regularization.

Limitations and Future Work

  • Annotation Sensitivity: The authors noted that for certain organs like the Esophagus (Es), performance was occasionally inconsistent. they attributed this to "inaccurate and inconsistent annotations" in the manual labeling process of the source datasets.
  • Computational Overhead: While not explicitly detailed as a failure, the two-phase approach (pre-training + clustering + multi-task training) is inherently more complex and computationally expensive than a single-stage SSL approach.
  • Future Directions: The authors suggest that further refining how subclasses are generated—perhaps through more sophisticated latent space modeling—could further push the boundaries of small organ detection.

Summary updated

Abstract

Semi-supervised learning (SSL) has shown notable potential in relieving the heavy demand of dense prediction tasks on large-scale well-annotated datasets, especially for the challenging multi-organ segmentation (MoS). However, the prevailing class-imbalance problem in MoS, caused by the substantial variations in organ size, exacerbates the learning difficulty of the SSL network. To alleviate this issue, we present a two-phase semi-supervised network (BSR-Net) with balanced subclass regularization for MoS. Concretely, in Phase I, we introduce a class-balanced subclass generation strategy based on balanced clustering to effectively generate multiple balanced subclasses from original biased ones according to their pixel proportions. Then, in Phase II, we design an auxiliary subclass segmentation (SCS) task within the multi-task framework of the main MoS task. The SCS task contributes a balanced subclass regularization to the main MoS task and transfers unbiased knowledge to the MoS network, thus alleviating the influence of the class-imbalance problem. Extensive experiments conducted on two publicly available datasets, i.e., the MICCAI FLARE 2022 dataset and the WORD dataset, verify the superior performance of our method compared with other methods.
BibTeX
@misc{feng2024alleviating,
  author = {Zhenghao Feng and Lu Wen and Binyu Yan and Jiaqi Cui and Yan Wang},
  title = {{Alleviating Class Imbalance in Semi-supervised Multi-organ Segmentation via Balanced Subclass Regularization}},
  year = {2024},
  eprint = {2408.14047},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2408.14047},
}
Figure