TL;DR
This study compares how different spatial representations—2D slices, 2.5D stacks, and 3D volumes—affect the performance of deep learning models in classifying lung nodule malignancy risk. Surprisingly, the "middle ground" 2.5D approach outperformed both single-slice 2D and full-volume 3D models, while common self-supervised pretraining techniques failed to provide a performance boost.
What Problem Does This Paper Address?
In medical imaging, specifically Computed Tomography (CT), researchers face a fundamental choice: how much spatial context should a model see? A CT scan is a 3D volume, but processing it as such is computationally expensive and requires vast amounts of labeled data.
Many researchers default to 2D slices for simplicity or 3D volumes for completeness. However, there is a middle ground—2.5D—where a few neighboring slices are stacked as input channels. This paper investigates whether the richer spatial information of 3D actually translates to better performance in a real-world clinical task: classifying the malignancy risk of pulmonary nodules. The authors also tackle the challenge of "indeterminate" nodules—those that radiologists can't clearly label as benign or malignant—which are often excluded from other studies but represent a significant portion of clinical reality.
Methods
The researchers implemented three distinct "tracks" based on the ResNet-18 architecture:
- 2D: Uses a single axial slice.
- 2.5D: Uses five neighboring slices stacked as channels.
- 3D: Uses a local volumetric crop of the CT scan.
For each track, they evaluated three training strategies:
- Model A (Baseline): Random initialization followed by standard supervised training.
- Model B (SSL): Initialization using Momentum Contrast (MoCo) self-supervised pretraining on a larger pool of unlabeled nodules, followed by fine-tuning.
- Model C (Augmentation): Random initialization with mild training-only augmentations (flips, rotations, intensity shifts).
The study design is summarized in Figure 1: Study design showing the three representation tracks and training recipes.
Datasets
The study utilized the LIDC-IDRI cohort, a standard public dataset for lung nodule analysis.
- Supervised Cohort: 2,654 nodules categorized into three classes (Benign, Indeterminate, Malignant) based on median radiologist ratings.
- SSL Pool: 5,133 nodules (including those without malignancy ratings) used for self-supervised pretraining.
- Splits: The data was split at the patient level (patient-disjoint) to ensure the model wasn't simply "memorizing" specific patients. The test set contained 396 nodules.
Key Insights and Ideas
- The 2.5D "Goldilocks" Effect: The 2.5D model achieved the highest performance. The authors hypothesize that a short stack of slices provides enough depth context to be useful without the optimization difficulties and high parameter-to-data ratio inherent in 3D volumes.
- SSL is Not a Silver Bullet: In all three tracks, MoCo pretraining actually decreased the Macro-F1 score compared to the baseline. This suggests that optimizing for "instance discrimination" (the goal of MoCo) doesn't necessarily create features that help distinguish between subtle clinical malignancy risks.
- The "Indeterminate" Hurdle: Including indeterminate nodules makes the task significantly harder. When the authors ran a secondary binary experiment (excluding indeterminate cases), performance jumped across all models (e.g., 2.5D Macro-F1 rose from 0.65 to 0.86). This highlights how much "performance" in medical AI depends on how you define the classes.
- Augmentation Wins: Simple, mild augmentations consistently improved results across all spatial representations, proving to be a more reliable regularizer than complex SSL schemes in this context.
Key Results
The primary metric was Macro-F1, which treats all classes equally regardless of how many samples they have.
| Input | Model C (Augmented) Macro-F1 | Accuracy |
|---|---|---|
| 2D | 0.6134 | 0.6128 |
| 2.5D | 0.6537 | 0.6869 |
| 3D | 0.5403 | 0.5455 |
As shown in Figure 2, the 2.5D model clearly leads in both the primary three-class task and the simplified binary task: Final test Macro-F1 results for three-class and binary tasks.
Limitations and Future Work
The authors are careful to note that their results don't prove 3D is "worse" in an absolute sense. The 3D model had fewer parameters and smaller batch sizes due to memory constraints, which may have hindered its optimization.
Key limitations include:
- Single Dataset: The study only used LIDC-IDRI; performance might vary on different hospital cohorts.
- Implementation Confounding: Differences in preprocessing (like 2.5D using mask-centered slices while 2D used a fixed slice) mean the results reflect the entire pipeline rather than just the dimensionality.
- Subjective Labels: The ground truth is based on radiologist opinions, not pathology (biopsy) results, which is the ultimate clinical gold standard.
Future work could involve harmonizing these pipelines further, exploring ordinal loss functions (since malignancy is a spectrum), and testing more advanced 3D architectures that might better handle the volumetric data.
Summary updated