Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images

Adam P. Harrison, Ziyue Xu, Kevin George, Le Lu, Ronald M. Summers, Daniel J. Mollura

Abstract

Pathological Lung Segmentation (PLS) is a critical precursor to automated pulmonary analysis, yet it remains a significant challenge due to the structural distortions caused by severe diseases. We present Progressive Holistically-Nested Networks (P-HNNs), an evolution of the HNN architecture that incorporates progressive multi-path connections to refine segmentation masks iteratively. By leveraging a VGG-16 backbone and a novel additive refinement strategy, P-HNN achieves a state-of-the-art Dice score of 0.985 on a large-scale multi-institutional dataset of 929 CT scans, demonstrating high robustness without increasing model complexity.

Introduction

Accurate lung segmentation is the essential first step in automated disease analysis, such as measuring lung volume or detecting tumors. However, "pathological" lungs often deviate significantly from normal anatomical models due to consolidations, effusions, or complete lung collapse. Traditional top-down methods that rely on anatomical landmarks or 3D shape priors frequently fail when a patient’s lung shape is atypical. Conversely, bottom-up pixel classification methods offer more flexibility but often struggle to capture fine details or produce clean masks because deep neural networks typically lose spatial resolution in deeper layers.

Methods: Progressive Multi-Path Connections

The researchers built upon the Holistically-Nested Network (HNN), a type of Fully Convolutional Network (FCN) that uses a VGG-16 backbone. The core innovation of P-HNN is the introduction of Progressive Multi-Path Connections to solve the output ambiguity found in standard HNNs.

Architecture of the P-HNN model illustrating the progressive refinement through multi-path connections.

Key architectural features include:

  1. Linked Stages: Instead of treating side-outputs as independent, P-HNN combines the activations of the current stage with the previous stage using simple addition before applying the final sigmoid function.
  2. Forced Refinement: Because activations can be positive or negative, this mathematical setup forces deeper layers to either "add to" or "subtract from" the previous prediction, effectively refining the mask at each step.
  3. Zero Overhead: This enhancement requires no additional convolutional layers or parameters, maintaining the model's simplicity and speed.
  4. Multi-Window Input: To ensure the network sees a wide range of tissue densities—from air to dense bone—the model processes 2D axial slices using three different Hounsfield Unit (HU) windows rescaled into a 3-channel image.

Datasets and Evaluation

The study utilized a massive collection of 929 CT scans from three diverse sources:

  • LTRC (Lung Tissue Research Consortium): 773 scans (Interstitial Lung Disease/COPD).
  • UHG (University Hospitals of Geneva): 73 scans.
  • NIH Clinical Center: 83 scans (Infection-based pathologies).

Notably, 848 of these scans are publicly available, making this one of the largest and most reproducible evaluations of PLS to date.

Key Results and Discussion

P-HNN achieved a mean Dice score of 0.985 and an Average Surface Distance (ASD) of 0.762 mm, significantly outperforming the standard HNN (0.978) and the previous non-deep-learning state-of-the-art (0.966).

Qualitative results comparing P-HNN (green) against ground truth and baseline methods (red/blue). Arrows indicate regions of significant improvement in pathological boundary detection.

Qualitatively, P-HNN proved much better at capturing fine details like pulmonary bullae (air pockets) and correctly excluding pleural effusions (fluid) that often confuse other models. The system is also highly efficient, segmenting an entire 3D volume in just 10–30 seconds on a single GPU. While the model occasionally makes minor anatomical errors, such as erroneously including the right main bronchus, its bottom-up robustness makes it a powerful tool for analyzing severely diseased lungs where shape priors fail.

Conclusion

The P-HNN framework demonstrates that progressive refinement through simple additive connections can significantly improve segmentation accuracy in complex medical imaging tasks without adding parameter overhead. Future work may explore the application of this framework to other organs and the integration of more 3D contextual information.

Summary updated

BibTeX
@inproceedings{harrison2017progressive,
  author = {Adam P. Harrison and Ziyue Xu and Kevin George and Le Lu and Ronald M. Summers and Daniel J. Mollura},
  title = {{Progressive and Multi-path Holistically Nested Neural Networks for Pathological Lung Segmentation from CT Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2017},
  doi = {10.1007/978-3-319-66179-7\_71},
}
Figure