PiaNet: A pyramid input augmented convolutional neural network for GGO detection in 3D lung CT scans

Weihua Liu, Xiabi Liu, Xiongbiao Luo, Murong Wang, Guangyuan Zheng, Guanghui HanarXiv 2020 · arXiv · OpenReview

TL;DR

This paper introduces PiaNet, a novel 3D convolutional neural network designed for the accurate detection of Ground-Glass Opacity (GGO) nodules in lung CT scans. PiaNet leverages a pyramid-input architecture with multiscale processing and a two-stage transfer learning strategy to achieve state-of-the-art performance, even with limited annotated data.

What Problem Does This Paper Address?

Lung cancer is a leading cause of cancer-related deaths, and early detection significantly improves patient outcomes. Ground-Glass Opacity (GGO) nodules are early indicators of lung cancer, but their indistinct boundaries and varied appearances make them challenging to detect, even for experienced radiologists. Existing computer-aided detection (CAD) systems often struggle with high false positive rates, sensitivity to GGO appearance variability, and difficulty in detecting very small or very large nodules. This paper aims to address these limitations by proposing a more robust and accurate GGO detection method.

Methods

PiaNet is a 3D convolutional neural network built upon a U-shaped architecture, comprising a feature-extraction module and a prediction module.

  • Feature-Extraction Module: This module employs a contracting pathway with "pyramid multiscale source connections." These connections inject downsampled versions of the input image at various scales into the contracting pathway, allowing the network to capture GGO information at different resolutions and prevent information loss. The expanding pathway uses skip connections to combine low-level and high-level features, preserving accurate location and multiscale information.
  • Prediction Module: This module utilizes a multiscale approach to simultaneously predict bounding boxes for GGO nodules and their classification confidence scores. It borrows the anchor box concept from Faster R-CNN, generating predictions at multiple scales and using multiple anchor sizes to accommodate the wide range of GGO sizes.
  • Two-Stage Transfer Learning: To overcome the challenge of limited annotated medical data, PiaNet is trained using a novel two-stage transfer learning strategy.
  • Stage 1: The feature-extraction module is first trained as part of a GGO classifier. This classifier is trained on a large dataset of GGO and non-GGO patches generated through extensive data augmentation from a small set of annotated CT scans. This stage helps learn robust features for distinguishing GGOs.
  • Stage 2: The pretrained feature-extraction module is then integrated into the full PiaNet architecture, and the entire network is fine-tuned using the annotated CT scans. This stage incorporates hard negative mining to balance the dataset and further improve generalization.

Datasets

The primary dataset used in this research is the LIDC-IDRI (Lung Image Database Consortium and Image Database Resource Initiative) dataset. This is described as the largest publicly available and most utilized dataset for lung nodule detection.

  • Data Preprocessing: CT scans were preprocessed by normalizing voxel size, intensity normalization, and lung parenchyma segmentation. The 3D volumes were then divided into 128x128x128 cubes using a sliding-window approach.
  • Training Data: A subset of 250 CT scans from the LIDC-IDRI dataset was used for training. Data augmentation techniques, including flipping, resizing, translating, rotating, and swapping, were extensively applied to generate a larger training set.
  • Test Data: The remaining 52 CT scans from the LIDC-IDRI dataset were used for testing.

Key Insights and Ideas

  • Pyramid Multiscale Source Connections: Introducing multiscale inputs directly into the contracting pathway is a novel way to improve feature extraction for objects of varying sizes, consistently boosting performance across different network architectures.
  • Two-Stage Transfer Learning for Medical Data: Transferring knowledge between related tasks within the same medical domain (GGO/non-GGO discrimination to GGO detection) proves more effective than traditional cross-domain transfer learning for limited medical datasets.
  • Multiscale Prediction is Crucial: The paper demonstrates that using multiscale feature maps in the prediction pathway is essential for covering the wide range of GGO sizes, with performance decreasing significantly when feature map scales are removed.
  • Complementary Subtasks: Dividing the GGO detection task into region location and region classification and training them jointly as a multi-task objective allows the subtasks to mutually reinforce each other, leading to better overall performance.

Key Results

PiaNet achieved state-of-the-art performance on the LIDC-IDRI dataset.

  • Sensitivity: PiaNet achieved a sensitivity of 91.75% with only one false positive per scan. In ablation studies, it reached 94.2% sensitivity at the lowest FPs/scan rate of 6.7.
  • Comparison with State-of-the-Art: PiaNet outperformed several leading methods and CAD systems, including GA-SSD, S4ND, SubsolidCAD, and Aidence.
  • It achieved a Competition Performance Metric (CPM) of 0.8826, significantly higher than GA-SSD (0.8547), S4ND (0.8666), SubsolidCAD (0.734), and Aidence (0.767).
  • Ablation Studies: The experiments confirmed the effectiveness of each component:
  • Source connections improved sensitivity by up to 6.9% for PiaNet.
  • The expanding pathway improved sensitivity from 78.2% (SSD) to 87.3% (PiaNet(-)).
  • Multiscale prediction improved sensitivity from 82.4% (U-Net) to 87.3% (PiaNet(-)).

Limitations and Future Work

  • Missed Nodules and False Positives: While PiaNet performs well, it still misses some very uncommon nodules or those with low contrast. False positives can occur when nodules are very close to similar-looking tissues like blood vessels or chest walls.
  • Future Work: The authors suggest focusing future research on training the network to better handle these hard cases by learning more discriminative features and emphasizing hard example mining during training.

Summary updated

Abstract

This paper proposes a new convolutional neural network with multiscale processing for detecting ground-glass opacity (GGO) nodules in 3D computed tomography (CT) images, which is referred to as PiaNet for short. PiaNet consists of a feature-extraction module and a prediction module. The former module is constructed by introducing pyramid multiscale source connections into a contracting-expanding structure. The latter module includes a bounding-box regressor and a classifier that are employed to simultaneously recognize GGO nodules and estimate bounding boxes at multiple scales. To train the proposed PiaNet, a two-stage transfer learning strategy is developed. In the first stage, the feature-extraction module is embedded into a classifier network that is trained on a large data set of GGO and non-GGO patches, which are generated by performing data augmentation from a small number of annotated CT scans. In the second stage, the pretrained feature-extraction module is loaded into PiaNet, and then PiaNet is fine-tuned using the annotated CT scans. We evaluate the proposed PiaNet on the LIDC-IDRI data set. The experimental results demonstrate that our method outperforms state-of-the-art counterparts, including the Subsolid CAD and Aidence systems and S4ND and GA-SSD methods. PiaNet achieves a sensitivity of 91.75% with only one false positive per scan
BibTeX
@misc{liu2020pianet,
  author = {Weihua Liu and Xiabi Liu and Xiongbiao Luo and Murong Wang and Guangyuan Zheng and Guanghui Han},
  title = {{PiaNet: A pyramid input augmented convolutional neural network for GGO detection in 3D lung CT scans}},
  year = {2020},
  eprint = {2009.05267},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.05267},
}
Figure