Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans

Jieneng Chen, Yingda Xia, Jiawen Yao, Ke Yan, Jianpeng Zhang, Le Lu, Fakai Wang, Bo Zhou, Mingyan Qiu, Qihang Yu, Mingze Yuan, Wei Fang, Yuxing Tang, Minfeng Xu, Jian Zhou, Yuqian Zhao, Qifeng Wang, Xianghua Ye, Xiaoli Yin, Yu Shi, Xin Chen, Jingren Zhou, Alan L. Yuille, Zaiyi Liu, Ling Zhang

TL;DR

This paper introduces CancerUniT, a novel unified AI model that can simultaneously detect, segment, and diagnose eight major types of cancer from CT scans. By employing a hierarchical query-based Transformer architecture, CancerUniT aims to mimic the comprehensive approach of human radiologists, moving towards a more efficient and accurate universal cancer screening tool.

What Problem Does This Paper Address?

Current AI systems for medical image analysis often focus on detecting or diagnosing a single type of cancer within a specific organ. This fragmented approach requires assembling multiple specialized models to replicate the workflow of a human radiologist, who routinely performs a full-body scan analysis for various diseases across multiple organs. This not only increases complexity but can also lead to inconsistencies and missed diagnoses. The paper addresses the need for a single, unified AI model that can handle the intricate task of detecting, segmenting, and diagnosing multiple cancers across different organs in a single pass, thereby improving clinical adoption and efficiency.

Methods

The core of CancerUniT is a Unified Tumor Transformer (CancerUniT) architecture. It leverages a query-based Mask Transformer approach, which is adapted for medical imaging. Key innovations include:

  • Hierarchical Query Representation: Object queries are decoupled into three groups: organ queries, tumor detection queries, and tumor diagnosis queries. These queries are organized hierarchically, reflecting the clinical process of first detecting a tumor and then diagnosing its specific subtype. This structure explicitly encourages the model to learn inter-organ and intra-organ relationships.
  • Dual-Task Query Decoding: To handle the inherent class exclusivity and overlapping nature of tumor subtypes, CancerUniT employs a dual-task decoding strategy. This separates the decoding process for tumor detection (identifying the presence and location of a tumor) and cancer diagnosis (classifying the specific tumor subtype), allowing for more accurate and distinct outputs.
  • UNet Backbone and Transformer Interaction: A UNet architecture is used for feature extraction from CT scans, providing multi-scale image features. These features are then fed into a Transformer decoder, where they interact with the hierarchical queries through multi-head cross-attention and self-attention mechanisms to refine the representations.

Datasets

The authors curated a large-scale dataset for training and evaluation:

  • CancerUniT Dataset: This dataset comprises 10,673 contrast-enhanced CT volumes from 10,042 patients. It includes eight major types of cancers (lung, colorectum, liver, stomach, breast, esophagus, pancreas, and kidney) and non-cancerous tumors. All tumors are pathology-confirmed, and 3D tumor masks were annotated by radiologists. This dataset is a significant contribution, offering a comprehensive collection for multi-cancer analysis.
  • Public Datasets: For evaluating generalizability, the model was also tested on three public single-tumor datasets (MSD pancreas, liver, and lung).

Key Insights and Ideas

  • Unified Approach Outperforms Specialization: The study strongly suggests that a single, unified model like CancerUniT can outperform both ensembles of single-organ expert models and existing multi-organ segmentation models in detection, segmentation, and diagnosis tasks. This highlights the power of holistic learning for complex medical imaging problems.
  • Hierarchical Representation is Clinically Inspired and Effective: The hierarchical organization of queries (organ -> detection -> diagnosis) directly mirrors how radiologists analyze scans. This clinically inspired design proves crucial for effectively capturing the complex relationships between different organs and tumor subtypes.
  • Transformer's Global Context is Key: The Transformer architecture's ability to capture global interactions between image patches and queries is vital for understanding the context of tumors within the broader anatomical landscape, leading to improved performance.
  • Semantic Segmentation is Superior for Detection: The paper argues that treating tumor detection as a semantic segmentation problem, rather than a traditional object detection problem, leads to better performance, particularly in reducing false positives.

Key Results

  • Superior Performance: CancerUniT significantly outperformed baseline models, including an 8-nnUNet ensemble, a unified nnUNet, TransUNet, and DeepLesion models, across patient-level detection, lesion-level detection, and voxel-level segmentation metrics.
  • Detection Accuracy: CancerUniT achieved a substantial improvement in average sensitivity and specificity for patient-level tumor detection, outperforming baselines by up to 21% in specificity.
  • Segmentation Quality: The model achieved the highest average Dice score for voxel-level tumor segmentation across various organs and tumor subtypes.
  • Diagnostic Accuracy: For tumor diagnosis, CancerUniT achieved the highest overall diagnosis performance (70.9%), surpassing even single-expert models.
  • Efficiency: CancerUniT is notably faster (4.5x) and lighter (8x fewer parameters) than an ensemble of single-tumor expert models.
  • Generalizability: The model demonstrated strong generalization capabilities on public single-tumor datasets without requiring additional training.

Limitations and Future Work

  • Data Size for Specific Subtypes: While the dataset is large, some specific tumor subtypes might still be underrepresented, potentially impacting performance for those rare cases.
  • Comparison with Blood Tests: The paper acknowledges that direct comparison with blood tests for cancer screening is challenging due to differing data modalities and validation protocols, though it suggests CT+AI offers a strong alternative.
  • Future Work: The authors envision CancerUniT as a step towards a universal high-performance cancer screening tool. Future work could involve expanding the range of detectable cancers and tumor subtypes, further refining the hierarchical representation, and exploring its application in different imaging modalities.

Summary updated

Abstract

Human readers or radiologists routinely perform full-body multi-organ multi-disease detection and diagnosis in clinical practice, while most medical AI systems are built to focus on single organs with a narrow list of a few diseases. This might severely limit AI's clinical adoption. A certain number of AI models need to be assembled non-trivially to match the diagnostic process of a human reading a CT scan. In this paper, we construct a Unified Tumor Transformer (UniT) model to detect (tumor existence and location) and diagnose (tumor characteristics) eight major cancer-prevalent organs in CT scans. UniT is a query-based Mask Transformer model with the output of multi-organ and multi-tumor semantic segmentation. We decouple the object queries into organ queries, detection queries and diagnosis queries, and further establish hierarchical relationships among the three groups. This clinically-inspired architecture effectively assists inter- and intra-organ representation learning of tumors and facilitates the resolution of these complex, anatomically related multi-organ cancer image reading tasks. UniT is trained end-to-end using a curated large-scale CT images of 10,042 patients including eight major types of cancers and occurring non-cancer tumors (all are pathology-confirmed with 3D tumor masks annotated by radiologists). On the test set of 631 patients, UniT has demonstrated strong performance under a set of clinically relevant evaluation metrics, substantially outperforming both multi-organ segmentation methods and an assembly of eight single-organ expert models in tumor detection, segmentation, and diagnosis. Such a unified multi-cancer image reading model (UniT) can significantly reduce the number of false positives produced by combined multi-system models. This moves one step closer towards a universal high-performance cancer screening tool.
BibTeX
@inproceedings{chen2023towards,
  author = {Jieneng Chen and Yingda Xia and Jiawen Yao and Ke Yan and Jianpeng Zhang and Le Lu and Fakai Wang and Bo Zhou and Mingyan Qiu and Qihang Yu and Mingze Yuan and Wei Fang and Yuxing Tang and Minfeng Xu and Jian Zhou and Yuqian Zhao and Qifeng Wang and Xianghua Ye and Xiaoli Yin and Yu Shi and Xin Chen and Jingren Zhou and Alan L. Yuille and Zaiyi Liu and Ling Zhang},
  title = {{Towards a Single Unified Model for Effective Detection, Segmentation, and Diagnosis of Eight Major Cancers Using a Large Collection of CT Scans}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01950},
}
Figure