Abstract
Medical image segmentation is a critical yet challenging task due to the inherent noise and irregular biological structures in clinical scans. While Convolutional Neural Networks (CNNs) excel at local feature extraction and Vision Transformers (ViTs) capture long-range dependencies, both have limitations in adaptability and computational efficiency. This paper introduces CiT-Net, a hybrid architecture that synergizes CNNs and ViTs. By leveraging Dynamic Deformable Convolutions (DDConv) and Cross-Dimensional Attention, CiT-Net effectively models irregular shapes and spatial-channel correlations. Experimental results on ISIC2018 and LiTS datasets demonstrate that CiT-Net achieves state-of-the-art performance with significantly fewer parameters and computational costs, all without the need for large-scale pre-training.
Introduction
Accurate segmentation of organs and lesions is fundamental to medical diagnosis and treatment planning. However, biological structures often exhibit high variability in shape and size, compounded by low contrast and noise in imaging modalities like CT and dermoscopy. Traditional CNN-based models, such as U-Net, rely on fixed-grid convolutions that lack the flexibility to adapt to irregular anatomical boundaries. Conversely, Vision Transformers offer powerful global context modeling but often overlook the intricate relationships between spatial dimensions and feature channels. Furthermore, Transformers are typically data-hungry and computationally intensive.
CiT-Net addresses these challenges by proposing a "hand-in-hand" dual-branch architecture. This design allows the CNN branch to provide local inductive biases while the Transformer branch captures global dependencies, creating a complementary system that is both robust and efficient.
Methods
The CiT-Net architecture consists of two parallel interactive branches: a Dynamically Adaptive CNN branch and a Cross-dimensional Fusion Transformer branch.
Key Innovations
- Dynamic Deformable Convolution (DDConv): To overcome the rigidity of standard convolutions, DDConv learns sampling offsets and weight coefficients. This enables the network to dynamically adjust its receptive field to match the specific morphology of the target structure.
- Shifted-Window Adaptive Complementary Attention Module ((S)W-ACAM): This module extends standard self-attention by incorporating branches that model correlations between spatial dimensions (height, width) and feature channels. This multi-dimensional approach improves the model's ability to distinguish targets from complex backgrounds.
- Lightweight Perceptron Module (LPM): Inspired by efficient architectures like GhostNet, the LPM replaces standard heavy MLP layers. This significantly reduces the model's parameter count and computational overhead without sacrificing performance.
- Interactive Feature Fusion: The two branches are not isolated; they exchange information throughout the encoding and decoding stages. This ensures that local geometric details from the CNN branch and global contextual information from the Transformer branch are effectively integrated.
Experiments and Results
CiT-Net was evaluated on the ISIC2018 skin lesion dataset and the LiTS liver tumor dataset. The model was trained from scratch, demonstrating its ability to learn effectively without ImageNet pre-training.
Performance and Efficiency
CiT-Net outperformed several state-of-the-art models, including TransUNet and Swin-Unet. On the ISIC2018 benchmark, the base version (CiT-Net-B) achieved a Dice Score (DI) of 91.23%. On the LiTS dataset, it reached a DI of 96.82%.
Beyond accuracy, CiT-Net demonstrates exceptional efficiency. The tiny version (CiT-Net-T) utilizes only 11.58 million parameters and 4.53 GFLOPs—roughly one-fourth the parameters of Swin-Unet and a fraction of TransUNet's computational cost. This efficiency makes it a strong candidate for clinical deployment on standard hardware.
Conclusion
CiT-Net successfully bridges the gap between CNNs and Vision Transformers for medical image segmentation. By introducing dynamic adaptability and cross-dimensional attention, the model achieves superior accuracy and efficiency. Future research will explore extending this architecture to native 3D data processing and validating its performance across a broader range of medical imaging modalities.
Summary updated