CovSegNet: A Multi Encoder-Decoder Architecture for Improved Lesion Segmentation of COVID-19 Chest CT Scans

Tanvir Mahmud, Md Awsafur Rahman, Shaikh Anowarul Fattah, Sun-Yuan Kung

Abstract

Automatic lung lesion segmentation of chest CT scans is a pivotal stage toward accurate diagnosis and severity measurement of COVID-19. Traditional U-shaped encoder-decoder architectures suffer from contextual information loss during pooling/upsampling and significant semantic gaps between encoded and decoded feature maps. Furthermore, processing 3D CT volumes directly introduces high computational complexity. This paper proposes CovSegNet, a novel architecture that utilizes horizontal expansion with multi-stage encoder-decoder modules and a two-phase training scheme. By employing a deep 2D network for ROI enhancement followed by a shallower 3D network for contextual refinement, CovSegNet achieves state-of-the-art performance with significantly reduced computational overhead.

Introduction

Manual inspection of 3D CT volumes for COVID-19 lesions is labor-intensive and error-prone. While deep learning, particularly the U-Net architecture, has advanced medical image segmentation, it faces three primary hurdles:

  1. Contextual Loss: Spatial details are often discarded during scaling operations.
  2. Semantic Gaps: Differences between low-level encoder features and high-level decoder features hinder convergence.
  3. 3D Complexity: Full 3D volume processing is memory-intensive and prone to vanishing gradients.

Methods: The CovSegNet Architecture

CovSegNet addresses these challenges through "horizontal" expansion, adding sequential encoder-decoder stages rather than just increasing vertical depth. This allows the model to refine its predictions across multiple passes.

The multi-stage CovSegNet architecture featuring horizontal expansion and multi-scale fusion.

Key Architectural Innovations

  • Multi-Stage Encoder-Decoder: The network consists of multiple sequential stages (e.g., Stage-1, Stage-2), facilitating parallel optimization and efficient gradient propagation.
  • Multi-Scale Fusion (MSF) & Pyramid Fusion (PF): Instead of standard skip connections, MSF modules aggregate features from all preceding levels. A Pyramid Fusion scheme then captures multi-scale context using various up-sampling and down-sampling windows to bridge semantic gaps.
  • Advanced Transition Units: Down and Up Transition units incorporate feature maps from all previous levels within a stage, creating a form of "transition memory" to recover spatial information typically lost during scaling.

Two-Phase Hybrid Training

To balance 3D contextual information with computational efficiency, the authors introduced a two-phase approach:

The two-phase training scheme: Phase-1 (2D ROI generation) and Phase-2 (3D refinement).
  1. Phase 1 (2D Training): A deep 2D network (CovSegNet2D) is trained on individual CT slices to generate Region of Interest (ROI) enhanced volumes.
  2. Phase 2 (3D Training): A shallower 3D network (CovSegNet3D) processes these ROI-enhanced volumes, capturing inter-slice depth information without the memory burden of a full-scale 3D model.

The training utilizes a Focal Tversky Loss function to address class imbalance, prioritizing "hard" examples and penalizing false negatives more heavily.

Experimental Results

The model was evaluated on two COVID-19 CT datasets and one non-medical drone dataset to test generalizability.

  • Accuracy: CovSegNet achieved a 94.1% Dice score on Dataset-1, outperforming U-Net, U-Net++, and V-Net by an average of 8.4%.
  • Efficiency: The CovSegNet2D-v2 variant used 94.8% fewer parameters than a standard U-Net while maintaining superior accuracy. The hybrid model (7.8M parameters) significantly undercut Unet3D (90.3M parameters).
  • Generalizability: The architecture also delivered top-tier results on the Semantic Drone Dataset, proving its robustness for general semantic segmentation tasks beyond medical imaging.

Discussion and Conclusion

The success of CovSegNet highlights that horizontal expansion and multi-scale feature integration are more effective than simple vertical deepening for complex medical segmentation. The hybrid 2D-3D training strategy provides a scalable solution for processing volumetric data. Future work will focus on larger, more diverse datasets and the integration of generative models for data augmentation to further enhance clinical utility in severity scoring and outcome prediction.

Summary updated

Abstract

Automatic lung lesions segmentation of chest CT scans is considered a pivotal stage towards accurate diagnosis and severity measurement of COVID-19. Traditional U-shaped encoder-decoder architecture and its variants suffer from diminutions of contextual information in pooling/upsampling operations with increased semantic gaps among encoded and decoded feature maps as well as instigate vanishing gradient problems for its sequential gradient propagation that result in sub-optimal performance. Moreover, operating with 3D CT-volume poses further limitations due to the exponential increase of computational complexity making the optimization difficult. In this paper, an automated COVID-19 lesion segmentation scheme is proposed utilizing a highly efficient neural network architecture, namely CovSegNet, to overcome these limitations. Additionally, a two-phase training scheme is introduced where a deeper 2D-network is employed for generating ROI-enhanced CT-volume followed by a shallower 3D-network for further enhancement with more contextual information without increasing computational burden. Along with the traditional vertical expansion of Unet, we have introduced horizontal expansion with multi-stage encoder-decoder modules for achieving optimum performance. Additionally, multi-scale feature maps are integrated into the scale transition process to overcome the loss of contextual information. Moreover, a multi-scale fusion module is introduced with a pyramid fusion scheme to reduce the semantic gaps between subsequent encoder/decoder modules while facilitating the parallel optimization for efficient gradient propagation. Outstanding performances have been achieved in three publicly available datasets that largely outperform other state-of-the-art approaches. The proposed scheme can be easily extended for achieving optimum segmentation performances in a wide variety of applications.
BibTeX
@article{mahmud2021covsegnet,
  author = {Tanvir Mahmud and Md Awsafur Rahman and Shaikh Anowarul Fattah and Sun-Yuan Kung},
  title = {{CovSegNet: A Multi Encoder-Decoder Architecture for Improved Lesion Segmentation of COVID-19 Chest CT Scans}},
  journal = {IEEE Trans. Artif. Intell.},
  year = {2021},
  url = {https://openreview.net/forum?id=9qLmVmLcAg},
}
Figure