Pulmonary Vessel Segmentation based on Orthogonal Fused U-Net++ of Chest CT Images

Hejie Cui, Xinglong Liu, Ning HuangMICCAI 2019 · DOI · OpenReview

TL;DR

This paper introduces a novel 2.5D convolutional neural network architecture, "Orthogonal Fused U-Net++," for segmenting pulmonary vessels in CT scans. By fusing information from three orthogonal views (axial, sagittal, coronal) and incorporating a post-processing step to refine vessel structures, the method achieves state-of-the-art accuracy with reduced computational cost compared to traditional 3D networks.

What Problem Does This Paper Address?

Accurate segmentation of pulmonary vessels in CT images is crucial for diagnosing and evaluating various lung diseases, such as pulmonary embolism. However, this task is inherently challenging due to the complex, intertwined nature of the vascular tree, coupled with image artifacts like noise and poor contrast in CT scans. Existing methods often struggle with this complexity, and while 3D networks can capture volumetric context, they demand significant computational resources. 2D networks, on the other hand, can miss vital contextual information across slices. This paper aims to bridge this gap by proposing a more efficient yet effective approach to capture the 3D structure of pulmonary vessels.

Methods

The core of the proposed method is an "Orthogonal Fused U-Net++" architecture. It leverages a 2.5D approach, meaning it processes a stack of adjacent slices (defined by a "slice radius") rather than individual slices or full 3D volumes. This allows the network to capture inter-slice context while maintaining lower computational complexity than 3D networks.

Key methodological innovations include:

  • 2.5D U-Net++: A U-Net++ architecture is adapted to process multiple adjacent slices. The "slice radius" parameter controls how many neighboring slices are convolved with the central slice, effectively incorporating local 3D context.
  • Orthogonal Fusion: The 2.5D U-Net++ is applied independently to three orthogonal axes (axial, sagittal, coronal). The feature maps from these three parallel networks are then fused. The paper explores different fusion strategies (intersection, union, average) and finds that average fusion yields the best results. This multi-planar fusion aims to create a more comprehensive volumetric representation.
  • Two-Stage Loss Function: To address the class imbalance inherent in vessel segmentation (where background voxels vastly outnumber vessel voxels), a two-stage training approach is used. An initial stage uses Negative Log Likelihood (NLL) loss for a coarse model, followed by a fine-tuned stage with a weight-balanced loss that specifically addresses the disproportionate representation of foreground and background voxels.
  • Vessel Structure Generation (Post-processing): After the initial segmentation, a post-processing step is employed. This involves generating a skeleton of the segmented vessel tree and then refining the segmentation by pruning small connected components and filling edges. This step leverages the inherent tree-like structure of vessels for improved accuracy.

Datasets

The study utilizes the publicly available Lung Image Database Consortium (LIDC) dataset. From this dataset, 300 cases of chest CTs were randomly selected. These were split into 270 cases for training and 30 cases for validation and testing. The ground truth masks were initially generated using an unsupervised method and then manually refined and validated by expert radiologists.

Key Insights and Ideas

  • The Power of 2.5D for Efficiency: The paper demonstrates that a well-designed 2.5D network can achieve performance comparable to, or even exceeding, more computationally expensive 3D networks for vessel segmentation. This is achieved by intelligently incorporating inter-slice context without the full volumetric overhead.
  • Multi-Planar Fusion is Key: Fusing information from orthogonal views (axial, sagittal, coronal) significantly enhances the segmentation accuracy. This highlights that understanding the vessel structure from multiple perspectives is crucial for robust segmentation.
  • Post-processing Refines Structure: Leveraging the inherent tree-like structure of pulmonary vessels through a skeletonization and pruning post-processing step is a powerful way to refine segmentation results and remove spurious detections.
  • Addressing Class Imbalance is Critical: The two-stage loss function, particularly the weight-balanced loss in the fine-tuning stage, is essential for handling the severe class imbalance in medical image segmentation tasks, leading to better performance.

Key Results

The proposed "Orthogonal Fused U-Net++" method achieved state-of-the-art results on the LIDC dataset.

  • Quantitative Performance: The method achieved a high average DICE score of 0.9272 and a precision of 0.9310.
  • Comparison: It significantly outperformed other state-of-the-art 2D and 3D network structures, including 2D U-Net++, 3D U-Net++, and 3D FCNs, as shown in Table 2. For instance, the average DICE score for the proposed method (0.9262) was substantially higher than 2D U-Net++ (0.6628) and 3D U-Net++ (0.7286).
  • Fusion Method Performance: Among the fusion methods tested, the "average" fusion strategy yielded the best results, outperforming intersection and union methods across various metrics.
  • Qualitative Results: Visual comparisons (Figure 2 and 3) show excellent agreement between the predicted segmentation and the ground truth, even for cases with varying image quality and vessel complexity.

Limitations and Future Work

  • Limitations: While the paper highlights the efficiency of the 2.5D approach, it doesn't explicitly detail the computational cost savings in terms of exact FLOPS or inference time compared to 3D networks. The reliance on expert-validated ground truth for training is standard but can be a bottleneck for larger datasets.
  • Future Work: The authors mention that the topological structure extracted during post-processing (edges, nodes, junction points) contains meaningful information for clinical practice. They plan to explore its application further, suggesting potential for more advanced clinical insights beyond just segmentation. They also suggest exploring its application in future work.

Summary updated

Abstract

Pulmonary vessel segmentation is important for clinical diagnosis of pulmonary diseases, while is also challenging due to the complicated structure. In this work, we present an effective framework and refinement process of pulmonary vessel segmentation from chest computed tomographic (CT) images. The key to our approach is a 2.5D segmentation network applied from three orthogonal axes, which presents a robust and fully automated pulmonary vessel segmentation result with lower network complexity and memory usage compared to 3D networks. The slice radius is introduced to convolve the adjacent information of the center slice and the multi-planar fusion optimizes the presentation of intra- and inter- slice features. Besides, the tree-like structure of the pulmonary vessel is extracted in the post-processing process, which is used for segmentation refining and pruning. In the evaluation experiments, three fusion methods are tested and the most promising one is compared with the state-of-the-art 2D and 3D structures on 300 cases of lung images randomly selected from LIDC dataset. Our method outperforms other network structures by a large margin and achieves by far the highest average DICE score of 0.9272 and precision of 0.9310, as per our knowledge from the pulmonary vessel segmentation models available in the literature.
BibTeX
@inproceedings{cui2019pulmonary,
  author = {Hejie Cui and Xinglong Liu and Ning Huang},
  title = {{Pulmonary Vessel Segmentation based on Orthogonal Fused U-Net++ of Chest CT Images}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2019},
  doi = {10.1007/978-3-030-32226-7\_33},
}
Figure