TL;DR
This paper introduces MIST, a novel medical image segmentation transformer that addresses the limitations of existing models in capturing both local and global contextual information. By employing a Convolutional Attention Mixing (CAM) decoder with a pre-trained MaxViT encoder, MIST achieves state-of-the-art performance on challenging medical imaging datasets.
What Problem Does This Paper Address?
Accurate segmentation of medical images is crucial for diagnosing diseases, planning treatments, and assessing their effectiveness. While deep learning models, particularly transformers, have shown great promise in this area due to their ability to capture long-range dependencies via self-attention, they often struggle with effectively incorporating local contextual information within images. Traditional Convolutional Neural Networks (CNNs), while good at local feature extraction, fall short in capturing these long-range dependencies. Existing transformer-based approaches, though improving, still face challenges in balancing local and global context, leading to potential inaccuracies in precise segmentation. This paper aims to bridge this gap by proposing a novel decoder architecture that explicitly integrates both local and global attention mechanisms.
Methods
The proposed MIST model consists of two main components:
- Encoder: A pre-trained Multi-Axis Vision Transformer (MaxViT) is used as the encoder. MaxViT is chosen for its ability to capture multi-axis attention, which is beneficial for processing image features.
- Decoder: The core innovation is the Convolutional Attention Mixing (CAM) decoder. This decoder is designed to aggregate attention computed at different stages. It comprises:
- Convolutional Projected Multi-Head Self-Attention (MSA): Instead of standard linear projection in MSA, the authors use convolutional projection. This reduces computational cost and preserves more spatial information, leading to better feature representation.
- Depth-wise (Shallow) Convolutions (SWC): These are used to increase the kernel's receptive field, enhancing the capture of both local and global spatial contexts. The paper explores varying dilation rates for these convolutions.
- Spatial Attention (SA) and Squeeze-and-Excitation Attention (SEA) Module (SSAM): These modules are integrated to further refine feature representation.
- Attention Mixing and Aggregation: The outputs from MSA and SSAM are mixed and aggregated, allowing the decoder to leverage dependencies from multiple attention mechanisms.
- Skip Connections: Standard skip connections are employed to merge multi-stage features from the encoder with the up-sampled decoder features, improving information flow and preserving semantic details.
Datasets
The MIST model was evaluated on two publicly available medical imaging datasets:
- ACDC (Automatic Cardiac Diagnosis Challenge): This dataset contains 100 cardiac MRI scans with segmentation masks for three organs: right ventricle (RV), myocardium (Myo), and left ventricle (LV).
- Synapse Multi-Organ Dataset: This dataset comprises 30 abdominal CT scans with segmentation masks for eight organs: aorta, gallbladder (GB), left kidney (KL), right kidney (KR), liver, pancreas (PC), spleen (SP), and stomach (SM).
Key Insights and Ideas
- Convolutional Projection for MSA: Replacing linear projection with convolutional projection in MSA is a key insight. It significantly reduces computational costs and parameter count while improving segmentation accuracy by better preserving spatial context.
- Attention Mixing for Comprehensive Context: The core idea of "attention mixing" in the CAM decoder, by aggregating outputs from self-attention, spatial attention, and squeeze-and-excitation attention, allows the model to effectively capture a richer set of dependencies at different spatial scales.
- Hierarchical Feature Fusion: The effective use of skip connections to merge encoder features with decoder features at multiple levels is crucial for maintaining high-resolution segmentation maps and preventing information loss.
- Balancing Local and Global: The combination of MaxViT's multi-axis attention encoder with the CAM decoder's attention-mixing strategy provides a robust solution for capturing both the fine-grained local details and the broader global context essential for medical image segmentation.
Key Results
The MIST model achieved state-of-the-art performance on both the ACDC and Synapse datasets:
- ACDC Dataset: MIST achieved a mean DICE score of 92.56±0.01, outperforming the previous best model (Parallel MERIT) by 0.24%. It also showed superior performance in segmenting individual organs.
- Synapse Multi-Organ Dataset: MIST achieved a mean DICE score of 86.92±0.08 and a mean HD95 of 11.08±0.07. This represents a significant improvement of over 4% in DICE score and 2% in HD95 compared to recent models like TransCASCADE and Parallel MERIT. The model demonstrated substantial gains across all segmented organs.
Limitations and Future Work
- Computational Cost of Concatenation: While concatenation of feature maps from dilated convolutions yields better performance, the authors note that it may result in higher computational costs.
- Future Work: The paper suggests that the MIST model with its CAM decoder has the potential to be effective in segmenting other medical image datasets, implying a broad applicability beyond the evaluated datasets. The authors also mention that their model has been made publicly available on GitHub, encouraging further research and development.
Summary updated