Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation

Jingnan Jia, Zhiwei Zhai, M. Els Bakker, Irene Hernández-Girón, Marius Staring, Berend C. StoelISBI 2021 · DOI

TL;DR

This paper introduces a novel multi-task semi-supervised learning framework for segmenting pulmonary lobes in CT scans. By cleverly combining lobe segmentation with auxiliary tasks like vessel segmentation and image reconstruction, and employing a focused alternating training strategy, the model significantly improves accuracy, even with limited annotated data.

What Problem Does This Paper Address?

Accurately segmenting pulmonary lobes in CT scans is crucial for analyzing lung diseases, as many conditions affect specific lobes. Traditional methods often struggle with accuracy, while deep learning approaches typically require vast amounts of meticulously annotated data, which is expensive and time-consuming to acquire. This paper tackles the challenge of achieving high-performance lobe segmentation when faced with limited annotated datasets, by leveraging multiple, potentially disparate, sources of information.

Methods

The core of the proposed method is a multi-task learning framework built upon a shared encoder. This framework comprises three subnets:

  • LobeNet: The primary segmentation network for identifying the five pulmonary lobes and background.
  • VesselNet: An auxiliary segmentation network for identifying pulmonary vessels and background.
  • ReconNet: A reconstruction network that learns to reconstruct the input image, acting as a form of self-supervision.

To effectively train these subnets, especially when their annotations come from different datasets, the authors propose a focused alternating training (FAT) strategy. Unlike a simple round-robin approach, FAT prioritizes the main task (lobe segmentation) by updating its gradients more frequently. Additionally, they incorporate multi-scale input (using patches at different resolutions) to capture both local and global context, and an adaptive learning rate mechanism to balance the learning of different tasks.

Datasets

The study utilizes a combination of datasets:

  • GLUCOLD and SSc: Internal datasets used for training and validation of lobe and vessel segmentation, respectively.
  • LUNA16: A dataset derived from LIDC, used for the unsupervised reconstruction task.
  • LOLA11: An independent dataset used for external testing and performance evaluation.

Key Insights and Ideas

  • Leveraging Auxiliary Tasks is Key: The paper strongly demonstrates that incorporating auxiliary tasks like vessel segmentation and image reconstruction significantly boosts the performance of the primary lobe segmentation task, even when these auxiliary tasks are trained on different datasets.
  • Focused Alternating Training (FAT) Matters: Simply alternating between tasks isn't enough. The FAT strategy, which prioritizes the main task, proves crucial for effectively balancing the learning process and achieving superior results compared to equally alternating training.
  • Multi-Scale Input and Adaptive Learning Rate are Synergistic: Combining multi-scale input with an adaptive learning rate further enhances performance, suggesting that capturing diverse contextual information and dynamically adjusting learning rates are vital for complex multi-task scenarios.
  • Generalizability of the Approach: The proposed multi-task training strategies are shown to be effective even when applied to a different backbone network (FRV-Net), indicating the generalizability of their method.

Key Results

The proposed multi-task semi-supervised model achieved significant improvements over single-task alternatives. On the GLUCOLD dataset, the mean surface distance (MSD) was reduced from 7.174 mm (single-task LobeNet) to 4.196 mm (multi-task LobeNet with FAT). Qualitative results also showed that the model could more precisely predict fissures and lung surfaces compared to baseline methods. The model also demonstrated competitive performance when compared to existing networks like FRV-Net.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, training multi-task models with multiple subnets and alternating strategies can be computationally intensive.
  • Dataset Specificity: The performance gains are demonstrated on specific datasets. Further validation on a wider variety of datasets would be beneficial.
  • Future Directions: The authors suggest that their approach, along with the training methods, shows potential for improving other deep learning architectures for medical image analysis. Further exploration into more sophisticated task balancing and data fusion techniques could be promising.

Summary updated

BibTeX
@inproceedings{jia2021multi,
  author = {Jingnan Jia and Zhiwei Zhai and M. Els Bakker and Irene Hernández-Girón and Marius Staring and Berend C. Stoel},
  title = {{Multi-task Semi-supervised Learning for Pulmonary Lobe Segmentation}},
  booktitle = {ISBI},
  year = {2021},
  doi = {10.1109/isbi48211.2021.9433985},
}
Figure