Deep Volumetric Universal Lesion Detection using Light-Weight Pseudo 3D Convolution and Surface Point Regression

Jinzheng Cai, Ke Yan, Chi-Tung Cheng, Jing Xiao, Chien-Hung Liao, Le Lu, Adam P. Harrison

Abstract

Identifying, measuring and reporting lesions accurately and comprehensively from patient CT scans are important yet time-consuming procedures for physicians. Computer-aided lesion/significant-findings detection techniques are at the core of medical imaging, which remain very challenging due to the tremendously large variability of lesion appearance, location and size distributions in 3D imaging. In this work, we propose a novel deep anchor-free one-stage VULD framework that incorporates (1) P3DC operators to recycle the architectural configurations and pre-trained weights from the off-the-shelf 2D networks, especially ones with large capacities to cope with data variance, and (2) a new SPR method to effectively regress the 3D lesion spatial extents by pinpointing their representative key points on lesion surfaces. Experimental validations are first conducted on the public large-scale NIH DeepLesion dataset where our proposed method delivers new state-of-the-art quantitative performance. We also test VULD on our in-house dataset for liver tumor detection. VULD generalizes well in both large-scale and small-sized tumor datasets in CT imaging.

Introduction

In medical imaging, CT scans are inherently 3D volumes. However, many automated detection tools treat them as a stack of independent 2D slices. This approach often fails to capture the necessary 3D context required to distinguish small or subtle lesions from healthy tissue. Current methodologies typically choose between efficient 2.5D networks, which lack full spatial understanding, or full 3D networks, which are computationally expensive and difficult to train on limited medical datasets.

Furthermore, standard detectors often rely on a "center-out" approach to predict bounding boxes. For tumors with vague boundaries, such as those found in liver cancer, this method frequently results in inaccurate spatial extents. This paper introduces the Volumetric Lesion Detector (VLD), a framework designed to bridge the gap between 2D efficiency and 3D context-awareness.

Methodology

The VLD framework introduces two primary innovations to improve detection accuracy and efficiency:

Light-Weight Pseudo 3D Convolution (P3DC)

To leverage the power of deep 2D architectures while maintaining volumetric awareness, the authors "inflate" a standard 2D DenseNet-121 using Axial-Coronal-Sagittal (ACS-3D) convolutions. This technique splits a standard 3D kernel into three 2D kernels oriented along different planes. This allows the model to "recycle" pre-trained weights from 2D networks (e.g., ImageNet), providing high capacity and spatial awareness without the memory overhead of full 3D convolutions.

Surface Point Regression (SPR)

Instead of estimating box dimensions solely from a center point, VLD employs a specialized head that regresses 16 points onto the lesion's surface. A triplet loss strategy is used to refine these points: features from the lesion center act as positive anchors, while features from the bounding box corners act as negative anchors. This forces the model to identify the transition zone, effectively "shrink-wrapping" the bounding box to the actual lesion surface.

The VLD framework architecture, showing the P3DC backbone and the specialized regression heads for center and surface points.

Experimental Results

Datasets and Evaluation

The model was evaluated on two benchmarks:

  1. NIH DeepLesion: A public dataset with over 32,000 annotated lesions across various organs.
  2. HCC Liver Dataset: An in-house dataset of 574 studies focusing on Hepatocellular Carcinoma, characterized by fuzzy boundaries in cirrhotic livers.

Quantitative Performance

VLD achieved new state-of-the-art (SOTA) performance on the NIH DeepLesion dataset. A significant breakthrough was observed in the detection of small lesions (<2cm), where sensitivity at 1 false positive per volume improved from 22.4% to 30.3%.

Sensitivity results on DeepLesion and HCC Liver datasets, highlighting the performance gains for small lesions and the impact of SPR.

On the challenging HCC dataset, the inclusion of Surface Point Regression (SPR) boosted sensitivity from 63.6% to 69.2%. Additionally, the P3DC backbone proved highly efficient, utilizing only 12.7M parameters—fewer than many 2.5D and 3D counterparts—while delivering superior accuracy.

Conclusion and Future Work

The VLD framework demonstrates that weight recycling via P3DC is a viable alternative to training 3D models from scratch. By focusing on lesion surfaces rather than just centers, the system significantly improves sensitivity for small, clinically vital tumors.

Future research may address the empirical tuning of axial-to-coronal resolution ratios and explore denser point sets for lesions with complex, non-spherical geometries. The current results suggest that VLD generalizes well across both large-scale universal datasets and specialized, small-sized tumor datasets.

Summary updated

Abstract

Identifying, measuring and reporting lesions accurately and comprehensively from patient CT scans are important yet time-consuming procedures for physicians. Computer-aided lesion/significant-findings detection techniques are at the core of medical imaging, which remain very challenging due to the tremendously large variability of lesion appearance, location and size distributions in 3D imaging. In this work, we propose a novel deep anchor-free one-stage VULD framework that incorporates (1) P3DC operators to recycle the architectural configurations and pre-trained weights from the off-the-shelf 2D networks, especially ones with large capacities to cope with data variance, and (2) a new SPR method to effectively regress the 3D lesion spatial extents by pinpointing their representative key points on lesion surfaces. Experimental validations are first conducted on the public large-scale NIH DeepLesion dataset where our proposed method delivers new state-of-the-art quantitative performance. We also test VULD on our in-house dataset for liver tumor detection. VULD generalizes well in both large-scale and small-sized tumor datasets in CT imaging.
BibTeX
@inproceedings{cai2020deep,
  author = {Jinzheng Cai and Ke Yan and Chi-Tung Cheng and Jing Xiao and Chien-Hung Liao and Le Lu and Adam P. Harrison},
  title = {{Deep Volumetric Universal Lesion Detection using Light-Weight Pseudo 3D Convolution and Surface Point Regression}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2020},
  doi = {10.1007/978-3-030-59719-1\_1},
}
Figure