A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations

Holger R. Roth, Le Lu, Ari Seff, Kevin M. Cherry, Joanne Hoffman, Shijun Wang, Jiamin Liu, Evrim Turkbey, Ronald M. Summers

Abstract

This paper introduces a novel "2.5D" representation for the automated detection of enlarged lymph nodes (LNs) in 3D Computed Tomography (CT) scans. By mapping orthogonal 2D slices into the RGB channels of a standard image and utilizing random sets of 3D transformations, the proposed method leverages the power of 2D Convolutional Neural Networks (CNNs) while maintaining essential spatial context. This approach significantly improves detection sensitivity and reduces false-positive rates compared to traditional 3D computer-aided detection (CADe) systems, all while avoiding the computational and data-intensive requirements of native 3D CNNs.

Introduction

Accurate detection of enlarged lymph nodes is a critical component of cancer staging and treatment monitoring. However, automating this process is challenging due to the low contrast between lymph nodes and surrounding tissues, as well as the high variability in their size, shape, and pose. Traditional 3D CADe systems often face a difficult trade-off between sensitivity and false-positive (FP) rates. While 3D CNNs offer a potential solution, they are frequently hampered by the "curse of dimensionality," requiring massive datasets and computational resources that are often unavailable in the medical domain.

Methodology

The authors propose a two-stage pipeline for lymph node detection. First, a high-sensitivity candidate generation stage identifies potential lymph nodes. Second, a 2.5D CNN-based classifier filters out false positives using a novel representation and augmentation strategy.

2.5D Representation and RGB Mapping

For each candidate Volume of Interest (VOI), the system extracts three orthogonal slices—axial, coronal, and sagittal—passing through the center. These slices are stacked into the Red, Green, and Blue (RGB) channels of a standard 2D image. This allows the model to utilize highly optimized 2D CNN architectures while capturing 3D spatial information.

Random Sets and Data Augmentation

To ensure robustness, the system does not rely on a single centered view. Instead, it generates a "random set" of observations by resampling the 3D VOI $N$ times (e.g., $N=100$) using random physical scales, 3D translations, and 3D rotations. These augmented patches are processed by a 2D CNN employing DropConnect for regularization.

The 2.5D representation pipeline: extracting orthogonal slices into RGB channels and augmenting with random 3D transformations.

Inference and Fusion

During the testing phase, the CNN evaluates all $N$ random views for a single candidate. The final classification probability is determined by averaging the predictions across the entire set, which serves to smooth noise and stabilize the final score.

Datasets and Experimental Setup

The method was evaluated on two distinct datasets:

  • Mediastinal LNs: 90 CT volumes with 388 labeled lymph nodes.
  • Abdominal LNs: 86 CT volumes with 595 labeled lymph nodes. Negative training examples were drawn from false-positive detections generated by the initial CADe stage that were more than 15mm away from true lymph nodes.

Results and Discussion

The 2.5D CNN approach demonstrated substantial improvements over previous state-of-the-art (SOTA) methods:

  • Mediastinum: Achieved 70% sensitivity at 3 FPs/volume (compared to the previous SOTA of 52.9% at 3.1 FPs/volume).
  • Abdomen: Achieved 83% sensitivity at 3 FPs/volume (compared to 70.5% at 13.0 FPs/volume).
  • Joint Training: Training the model on both datasets simultaneously improved mediastinal sensitivity by approximately 10%, suggesting that cross-domain features enhance robustness.

The system is also computationally efficient, requiring only about 5 minutes per CT volume for classification on a standard GPU.

Conclusion

The proposed 2.5D representation effectively bridges the gap between 2D efficiency and 3D spatial context. By aggregating predictions from random sets of views, the system achieves a powerful ensemble effect that mitigates overfitting and improves clinical viability. This framework shows great promise for adaptation to other 3D medical imaging tasks beyond lymph node detection.

Summary updated

Abstract

Automated Lymph Node (LN) detection is an important clinical diagnostic task but very challenging due to the low contrast of surrounding structures in Computed Tomography (CT) and to their varying sizes, poses, shapes and sparsely distributed locations. State-of-the-art studies show the performance range of 52.9% sensitivity at 3.1 false-positives per volume (FP/vol.), or 60.9% at 6.1 FP/vol. for mediastinal LN, by one-shot boosting on 3D HAAR features. In this paper, we first operate a preliminary candidate generation stage, towards 100% sensitivity at the cost of high FP levels (40 per patient), to harvest volumes of interest (VOI). Our 2.5D approach consequently decomposes any 3D VOI by resampling 2D reformatted orthogonal views N times, via scale, random translations, and rotations with respect to the VOI centroid coordinates. These random views are then used to train a deep Convolutional Neural Network (CNN) classifier. In testing, the CNN is employed to assign LN probabilities for all N random views that can be simply averaged (as a set) to compute the final classification probability per VOI. We validate the approach on two datasets: 90 CT volumes with 388 mediastinal LNs and 86 patients with 595 abdominal LNs. We achieve sensitivities of 70%/83% at 3 FP/vol. and 84%/90% at 6 FP/vol. in mediastinum and abdomen respectively, which drastically improves over the previous state-of-the-art work.
BibTeX
@inproceedings{roth2014new,
  author = {Holger R. Roth and Le Lu and Ari Seff and Kevin M. Cherry and Joanne Hoffman and Shijun Wang and Jiamin Liu and Evrim Turkbey and Ronald M. Summers},
  title = {{A New 2.5D Representation for Lymph Node Detection Using Random Sets of Deep Convolutional Neural Network Observations}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2014},
  doi = {10.1007/978-3-319-10404-1\_65},
}
Figure