Abstract
Accurate brain tumor segmentation in multi-modal MRI is essential for clinical diagnosis and treatment planning. However, standard convolutional neural networks (CNNs) often struggle with low tissue contrast and long-range dependencies, while standard self-attention mechanisms suffer from prohibitive computational costs in 3D volumes. This paper presents the Category Guided Attention U-Net (CGA U-Net), which introduces a supervised attention mechanism designed to capture global context efficiently. By explicitly supervising attention maps and optimizing for class separation, the CGA U-Net achieves state-of-the-art performance on the BraTS 2019 benchmark while significantly reducing parameters and FLOPs compared to established models like nnU-Net.
Introduction
Automatic brain tumor segmentation is a challenging task due to the high variability in tumor size, shape, and location, as well as the inherent "fuzzy" boundaries between pathological and healthy tissues. While U-Net architectures excel at capturing local patterns, their limited receptive fields often fail to model the global spatial relationships necessary for precise segmentation.
To address this, researchers have explored non-local networks and self-attention. However, traditional self-attention scales quadratically ($O(N^2)$) with the number of pixels, making it computationally expensive for high-resolution 3D medical images. Furthermore, unsupervised attention maps can be noisy and lack anatomical interpretability. The CGA U-Net addresses these limitations by replacing standard self-attention with a Supervised Attention Module (SAM) that leverages ground-truth guidance to learn stable, category-specific features.
Methodology
The CGA U-Net architecture integrates three core components designed for efficiency and accuracy: the Supervised Attention Module (SAM), Intra-class Update, and Inter-class Distance optimization.
Supervised Attention Module (SAM)
Unlike standard self-attention, which computes a massive similarity matrix, SAM utilizes a dedicated "Attention Conv Path" to generate attention maps directly from the input image. These maps are explicitly supervised by ground-truth labels during training ($L_A$ loss). This ensures that the attention mechanism focuses on relevant anatomical structures and category-specific spatial features from the outset.
Intra-class Update and Inter-class Distance
Once the attention map identifies potential category regions (e.g., edema, tumor core), the network calculates a "prototype vector" for each class. The Intra-class Update then reconstructs the feature map by aggregating pixels belonging to the same category, effectively refining noisy local features. To further improve discriminative power, an Inter-class Distance loss is introduced to maximize the Euclidean distance between these prototype vectors, preventing the model from confusing similar-looking tissues.
As shown in the comparison above, the proposed SAM (c) avoids the $HW \times HW$ matrix operations of standard self-attention (a) and the complex reshaping of the CP layer (b), instead focusing on a class-based spatial map that is directly supervised.
Experimental Results
The model was evaluated on the BraTS 2019 dataset, which includes 3D MRI scans for 335 training and 125 validation patients. The task involves segmenting the Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC).
Segmentation Performance
The CGA U-Net achieved competitive Dice scores across all tumor regions:
- Enhancing Tumor (ET): 78.83%
- Whole Tumor (WT): 89.29%
- Tumor Core (TC): 82.32%
Computational Efficiency
A key advantage of the CGA U-Net is its efficiency. Compared to the popular nnU-Net, the CGA U-Net requires significantly fewer resources:
- CGA U-Net: 4.77M parameters / 151.12G FLOPs
- nnU-Net: 31.2M parameters / 475.7G FLOPs
Qualitative analysis demonstrated that the SAM module effectively "repairs" missing tumor details and prevents over-segmentation in low-contrast regions, providing more anatomically plausible results.
Conclusion
The Category Guided Attention U-Net demonstrates that explicit supervision of attention mechanisms can lead to both higher accuracy and greater computational efficiency in medical image segmentation. By shifting from unsupervised $O(N^2)$ similarity matrices to category-guided spatial maps, the model captures global context without the typical overhead. Future work will focus on testing the generalizability of this approach across larger, more diverse datasets and exploring its scalability to tasks with a higher number of semantic categories.
Summary updated