DeepPulmoTB: A benchmark dataset for multi-task learning of tuberculosis lesions in lung computerized tomography (CT)

Zhuoyi Tan, Hizmawati Madzin, Norafida Bahari, Yang Chongshuang, Wei Sun, Tianyu Nie, Fengzhou CaiHeliyon 2024 · DOI

TL;DR

This paper introduces DeepPulmoTB, a novel benchmark dataset for tuberculosis (TB) detection in CT scans, featuring multi-task annotations for both lesion segmentation and attribute classification. They also propose DeepPulmoTBNet (DPTBNet), a multi-task learning model that leverages an improved iterative refinement strategy to achieve state-of-the-art performance in segmenting and classifying TB lesions.

What Problem Does This Paper Address?

Tuberculosis (TB) remains a major global health concern, and accurate early diagnosis is crucial for effective treatment and improved patient outcomes. Computer-aided diagnosis (CAD) tools, particularly those based on deep learning, show great promise in assisting with TB diagnosis. However, the development of these tools is severely hampered by a lack of comprehensive, well-annotated datasets for CT scans. Existing datasets often have limited annotation categories for segmentation and may not support joint tasks like segmentation and classification. This scarcity of data limits the ability of deep learning models to learn complex TB lesion features, hindering the development of robust CAD systems.

Methods

The core of this work lies in the introduction of DeepPulmoTBNet (DPTBNet), a novel multi-task learning framework designed to simultaneously perform semantic segmentation of TB lesions and classify lung cavity attributes.

  • Multi-task Architecture: DPTBNet utilizes SwinUnetR as its backbone for feature extraction, leveraging its strong performance in 3D medical image segmentation. The architecture is designed to share features between the segmentation and classification tasks, allowing them to benefit from each other. A classification branch is integrated into the decoder of SwinUnetR.
  • Multi-scale Feature Fusion: To better capture diverse TB lesion features, DPTBNet employs a multi-scale feature fusion strategy for the classification task. Features from different stages of the SwinUnetR decoder are combined to provide a richer representation.
  • Improved Iterative Refinement: A key innovation is an enhanced iterative optimization algorithm. This approach refines feature maps by iteratively incorporating segmentation probability maps from previous iterations. This helps the model focus on challenging lesion areas with unclear boundaries, improving segmentation accuracy. The algorithm also incorporates gradient clipping for training stability and adaptive thresholding.

Datasets

The paper introduces the DeepPulmoTB dataset, a new benchmark specifically designed for TB diagnosis using CT scans.

  • Data Source: Images are collected from the ImageCLEF TB series.
  • Annotations: DeepPulmoTB offers comprehensive annotations for two tasks:
  • Multi-category Semantic Segmentation: Annotations include "lung cavity," "Consolidation and Lung Cavity Wall (C-LCW)," and "lung areas." The C-LCW category is a novel combination to address the difficulty of distinguishing these similar-density regions in non-enhanced CTs.
  • Lung Cavity Attribute Classification: Annotations include the number and size of lung cavities.
  • Exclusions: Miliary tuberculosis, endobronchial tuberculosis, and tuberculous pleurisy (without parenchymal lesions) were excluded to focus on more common pulmonary TB forms and ensure annotation consistency.
  • Comparison: DeepPulmoTB is highlighted as the first multi-task dataset for joint TB lesion classification and segmentation and the first multi-category TB lesion segmentation dataset.

Key Insights and Ideas

  • The Power of Multi-task Learning for TB: The study demonstrates that jointly learning segmentation and classification tasks significantly improves performance compared to single-task approaches, especially in data-scarce domains like TB diagnosis.
  • Iterative Refinement for Ambiguous Boundaries: The iterative feature refinement strategy is crucial for handling the inherent ambiguity and unclear boundaries of TB lesions in CT scans, leading to more precise segmentation.
  • Multi-scale Features Capture Diverse Information: Fusing features from different scales of the network effectively captures both global context and local details, which is essential for accurate classification of TB lesion attributes.
  • Dataset Design Matters: The creation of DeepPulmoTB, with its specific multi-category segmentation and attribute classification annotations, directly addresses the limitations of existing datasets and enables the development of more sophisticated CAD tools.

Key Results

  • Segmentation Performance: DPTBNet achieved state-of-the-art segmentation performance on the DeepPulmoTB dataset, outperforming baseline models and other ablation variants across metrics like 95HD, DSC, and IoU. For instance, DPTBNet achieved a DSC of 69.6% and an IoU of 55.8% with optimal parameters.
  • Classification Performance: DPTBNet also demonstrated strong classification performance, achieving an ACC of 0.757, REC of 0.804, PRE of 0.817, FPR of 0.408, and F1 of 0.810. The multi-task approach consistently improved classification metrics compared to single-task baselines.
  • Iterative Refinement Benefits: The improved iterative algorithm (IIA) consistently enhanced segmentation performance, with DPTBNet showing DSC improvements of up to 7.8% compared to its non-iterative counterpart.
  • Multi-scale Fusion Advantage: The multi-scale feature fusion for classification also led to improved results, with DPTBNet achieving the best ACC, PRE, FPR, and F1 scores when utilizing features from all four branches.

Limitations and Future Work

  • Dataset Size: While comprehensive, the DeepPulmoTB dataset, like many medical imaging datasets, could benefit from further expansion to include more diverse cases and potentially larger volumes.
  • Generalizability: While the model shows strong performance on the DeepPulmoTB dataset, further validation on external datasets would be beneficial to assess its generalizability to different imaging protocols and patient populations.
  • Computational Cost: The iterative refinement process, while effective, adds to the computational cost of training and inference. Future work could explore more efficient iterative strategies or model compression techniques.
  • Clinical Integration: The paper focuses on technical performance. Future work could explore the integration of DPTBNet into clinical workflows and evaluate its impact on diagnostic efficiency and accuracy in real-world settings.

Summary updated

Abstract

Tuberculosis (TB) remains a significant global health challenge, characterized by high incidence and mortality rates on a global scale. With the rapid advancement of computer-aided diagnosis (CAD) tools in recent years, CAD has assumed an increasingly crucial role in supporting TB diagnosis. Nonetheless, the development of CAD for TB diagnosis heavily relies on well-annotated computerized tomography (CT) datasets. Currently, the available annotations in TB CT datasets are still limited, which in turn restricts the development of CAD tools for TB diagnosis to some extent. To address this limitation, we introduce DeepPulmoTB, a CT multi-task learning dataset explicitly designed for TB diagnosis. To demonstrate the advantages of DeepPulmoTB, we propose a novel multi-task learning model, DeepPulmoTBNet (DPTBNet), for the joint segmentation and classification of lesion tissues in CT images. The architecture of DPTBNet comprises two subnets: SwinUnetR for the segmentation task, and a lightweight multi-scale network for the classification task. Furthermore, to enhance the model's capacity to capture TB lesion features, we introduce an improved iterative optimization algorithm that refines feature maps by integrating probability maps obtained in previous iterations. Extensive experiments validate the effectiveness of DPTBNet and the practicality of the DeepPulmoTB dataset.
BibTeX
@inproceedings{tan2024deeppulmotb,
  author = {Zhuoyi Tan and Hizmawati Madzin and Norafida Bahari and Yang Chongshuang and Wei Sun and Tianyu Nie and Fengzhou Cai},
  title = {{DeepPulmoTB: A benchmark dataset for multi-task learning of tuberculosis lesions in lung computerized tomography (CT)}},
  booktitle = {Heliyon},
  year = {2024},
  doi = {10.1016/j.heliyon.2024.e25490},
}
Figure