CAE-Transformer: Transformer-based Model to Predict Invasiveness of Lung Adenocarcinoma Subsolid Nodules from Non-thin Section 3D CT Scans

Shahin Heidarian, Parnian Afshar, Anastasia Oikonomou, Konstantinos N. Plataniotis, Arash MohammadiarXiv 2021 · OpenReview

TL;DR

This paper introduces the CAE-Transformer, a novel deep learning framework that uses a Convolutional Auto-Encoder (CAE) and a Transformer architecture to predict the invasiveness of lung adenocarcinoma subsolid nodules from standard CT scans. By effectively capturing global inter-slice relationships without requiring precise nodule annotations, the CAE-Transformer achieves high accuracy, offering a more timely and less subjective alternative to current diagnostic methods.

What Problem Does This Paper Address?

Lung cancer remains a leading cause of mortality, with Lung Adenocarcinoma (LUAC) being the most prevalent type. Accurately determining the invasiveness of LUAC subsolid nodules (SSNs) is crucial for timely treatment planning. Currently, this often relies on subjective radiologist interpretation of CT scans or invasive surgical resection for pathological confirmation. While radiologists' analysis is time-consuming and prone to error, surgical resection isn't always a viable or timely solution. Existing automated methods often require thin-slice CT scans or meticulously annotated nodule regions, which are not always available in routine clinical practice. This paper aims to bridge this gap by developing a robust framework that can predict invasiveness from standard, non-thin section CT scans with minimal radiologist supervision and without requiring precise nodule annotations.

Methods

The core of the proposed solution is the CAE-Transformer framework, which combines a Convolutional Auto-Encoder (CAE) with a Transformer architecture.

  1. Convolutional Auto-Encoder (CAE): This component is used for unsupervised feature extraction. It takes CT slices as input and learns to compress them into informative feature maps. The CAE is pre-trained on a large public dataset (LIDC-IDRI) and then fine-tuned on the specific in-house dataset. This step aims to generate rich, compressed representations of the CT slices.
  2. Transformer Encoder: The feature maps generated by the CAE from individual CT slices are then fed into a modified Transformer encoder. The Transformer's strength lies in its Multi-Head Self-Attention mechanism, which allows it to capture global relationships and dependencies between these feature maps (representing inter-slice relations). This is a key innovation, as traditional sequential models struggle with long-range dependencies and global context.
  3. Modifications for Small Datasets: Recognizing that medical datasets are often small, the authors made specific modifications to the standard Transformer architecture. These include removing the class token and replacing the Global Average Pooling (GAP) layer with a Global Max Pooling (GMP) layer. They also employed label smoothing during training and only pre-trained the auto-encoder part, not the entire Transformer.
  4. Lung Segmentation: As a pre-processing step, a U-Net-based model is used to extract lung parenchyma from CT scans, removing distracting components and reducing computational complexity.

Datasets

  • LIDC-IDRI Dataset: Used for pre-training the Convolutional Auto-Encoder (CAE). This is a large, publicly available dataset of CT scans with annotated lung nodules.
  • In-house Dataset: A dataset of 114 pathologically proven Sub-Solid Nodules (SSNs) from non-thin section 3D CT scans. This dataset was initially introduced in a previous study and augmented with five additional cases. The nodules are classified into pre-invasive/minimally invasive (benign) and invasive (malignant) categories.

Key Insights and Ideas

  • Transformer for Inter-Slice Relations: The paper highlights the power of Transformer architectures, specifically their self-attention mechanism, in capturing global relationships between CT slices. This is crucial for understanding the 3D context of lung nodules, which is often missed by slice-by-slice or purely local methods.
  • Unsupervised Feature Extraction with CAE: Using a CAE for unsupervised feature extraction from CT slices is an effective way to generate informative representations without relying on manual annotations, which are labor-intensive and subjective.
  • Adaptation for Small Medical Datasets: The modifications made to the Transformer architecture (e.g., removing class token, using GMP, targeted pre-training) demonstrate a practical approach to applying powerful models like Transformers to data-scarce medical imaging domains.
  • Eliminating Precise Nodule Annotation: A significant contribution is the ability to predict invasiveness using whole CT images, bypassing the need for precise nodule segmentation or annotation, which simplifies the data preparation pipeline and makes the model more clinically applicable.

Key Results

The CAE-Transformer framework achieved state-of-the-art performance on their in-house dataset using 10-fold cross-validation:

  • Accuracy: 87.73%
  • Sensitivity: 88.67%
  • Specificity: 86.33%
  • AUC: 0.913

These results significantly outperform the original radiomics-based model from Reference [7] and other deep learning-based counterparts, including CAE-LSTM and 3D-CNN. The paper also notes that larger, deeper networks did not perform as well on their small dataset, underscoring the importance of their adapted architecture and training strategies.

Limitations and Future Work

  • Dataset Size: While the framework is designed to work with smaller datasets, the authors acknowledge that achieving clinically applicable deep learning solutions requires more extensive experiments and larger, more diverse datasets.
  • Future Work: The authors plan to:
  • Increase the size and diversity of the dataset.
  • Target a three-way SSN classification task (pre-invasive, minimally invasive, invasive).
  • Investigate the incorporation of radiomics and morphological features into the CAE-Transformer framework.
  • Collaborate with medical centers to further refine and validate the model.

Summary updated

Abstract

Lung cancer is the leading cause of mortality from cancer worldwide and has various histologic types, among which Lung Adenocarcinoma (LUAC) has recently been the most prevalent one. The current approach to determine the invasiveness of LUACs is surgical resection, which is not a viable solution to fight lung cancer in a timely fashion. An alternative approach is to analyze chest Computed Tomography (CT) scans. The radiologists' analysis based on CT images, however, is subjective and might result in a low accuracy. In this paper, a transformer-based framework, referred to as the "CAE-Transformer", is developed to efficiently classify LUACs using whole CT images instead of finely annotated nodules. The proposed CAE-Transformer can achieve high accuracy over a small dataset and requires minor supervision from radiologists. The CAE Transformer utilizes an encoder to automatically extract informative features from CT slices, which are then fed to a modified transformer to capture global inter-slice relations and provide classification labels. Experimental results on our in-house dataset of 114 pathologically proven Sub-Solid Nodules (SSNs) demonstrate the superiority of the CAE-Transformer over its counterparts, achieving an accuracy of 87.73%, sensitivity of 88.67%, specificity of 86.33%, and AUC of 0.913, using a 10-fold cross-validation.
BibTeX
@misc{heidarian2021cae,
  author = {Shahin Heidarian and Parnian Afshar and Anastasia Oikonomou and Konstantinos N. Plataniotis and Arash Mohammadi},
  title = {{CAE-Transformer: Transformer-based Model to Predict Invasiveness of Lung Adenocarcinoma Subsolid Nodules from Non-thin Section 3D CT Scans}},
  year = {2021},
  eprint = {2110.08721},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2110.08721},
}
Figure