Category Guided Attention Network for Brain Tumor Segmentation in MRI

Jiangyun Li, Hong Yu, Chen Chen, Meng Ding, Sen ZhaarXiv 2022 · DOI · arXiv

Abstract

Accurate brain tumor segmentation in multi-modal MRI is essential for clinical diagnosis and treatment planning. However, standard convolutional neural networks (CNNs) often struggle with low tissue contrast and long-range dependencies, while standard self-attention mechanisms suffer from prohibitive computational costs in 3D volumes. This paper presents the Category Guided Attention U-Net (CGA U-Net), which introduces a supervised attention mechanism designed to capture global context efficiently. By explicitly supervising attention maps and optimizing for class separation, the CGA U-Net achieves state-of-the-art performance on the BraTS 2019 benchmark while significantly reducing parameters and FLOPs compared to established models like nnU-Net.

Introduction

Automatic brain tumor segmentation is a challenging task due to the high variability in tumor size, shape, and location, as well as the inherent "fuzzy" boundaries between pathological and healthy tissues. While U-Net architectures excel at capturing local patterns, their limited receptive fields often fail to model the global spatial relationships necessary for precise segmentation.

To address this, researchers have explored non-local networks and self-attention. However, traditional self-attention scales quadratically ($O(N^2)$) with the number of pixels, making it computationally expensive for high-resolution 3D medical images. Furthermore, unsupervised attention maps can be noisy and lack anatomical interpretability. The CGA U-Net addresses these limitations by replacing standard self-attention with a Supervised Attention Module (SAM) that leverages ground-truth guidance to learn stable, category-specific features.

Methodology

The CGA U-Net architecture integrates three core components designed for efficiency and accuracy: the Supervised Attention Module (SAM), Intra-class Update, and Inter-class Distance optimization.

Overview of the CGA U-Net architecture featuring the Supervised Attention Module (SAM) and the Attention Conv Path.

Supervised Attention Module (SAM)

Unlike standard self-attention, which computes a massive similarity matrix, SAM utilizes a dedicated "Attention Conv Path" to generate attention maps directly from the input image. These maps are explicitly supervised by ground-truth labels during training ($L_A$ loss). This ensures that the attention mechanism focuses on relevant anatomical structures and category-specific spatial features from the outset.

Intra-class Update and Inter-class Distance

Once the attention map identifies potential category regions (e.g., edema, tumor core), the network calculates a "prototype vector" for each class. The Intra-class Update then reconstructs the feature map by aggregating pixels belonging to the same category, effectively refining noisy local features. To further improve discriminative power, an Inter-class Distance loss is introduced to maximize the Euclidean distance between these prototype vectors, preventing the model from confusing similar-looking tissues.

Comparison between (a) Self-Attention Module, (b) CP Layer, and (c) the proposed Supervised Attention Module.

As shown in the comparison above, the proposed SAM (c) avoids the $HW \times HW$ matrix operations of standard self-attention (a) and the complex reshaping of the CP layer (b), instead focusing on a class-based spatial map that is directly supervised.

Experimental Results

The model was evaluated on the BraTS 2019 dataset, which includes 3D MRI scans for 335 training and 125 validation patients. The task involves segmenting the Enhancing Tumor (ET), Whole Tumor (WT), and Tumor Core (TC).

Segmentation Performance

The CGA U-Net achieved competitive Dice scores across all tumor regions:

  • Enhancing Tumor (ET): 78.83%
  • Whole Tumor (WT): 89.29%
  • Tumor Core (TC): 82.32%

Computational Efficiency

A key advantage of the CGA U-Net is its efficiency. Compared to the popular nnU-Net, the CGA U-Net requires significantly fewer resources:

  • CGA U-Net: 4.77M parameters / 151.12G FLOPs
  • nnU-Net: 31.2M parameters / 475.7G FLOPs

Qualitative analysis demonstrated that the SAM module effectively "repairs" missing tumor details and prevents over-segmentation in low-contrast regions, providing more anatomically plausible results.

Conclusion

The Category Guided Attention U-Net demonstrates that explicit supervision of attention mechanisms can lead to both higher accuracy and greater computational efficiency in medical image segmentation. By shifting from unsupervised $O(N^2)$ similarity matrices to category-guided spatial maps, the model captures global context without the typical overhead. Future work will focus on testing the generalizability of this approach across larger, more diverse datasets and exploring its scalability to tasks with a higher number of semantic categories.

Summary updated

Abstract

Objective: Magnetic resonance imaging (MRI) has been widely used for the analysis and diagnosis of brain diseases. Accurate and automatic brain tumor segmentation is of paramount importance for radiation treatment. However, low tissue contrast in tumor regions makes it a challenging task.Approach: We propose a novel segmentation network named Category Guided Attention U-Net (CGA U-Net). In this model, we design a Supervised Attention Module (SAM) based on the attention mechanism, which can capture more accurate and stable long-range dependency in feature maps without introducing much computational cost. Moreover, we propose an intra-class update approach to reconstruct feature maps by aggregating pixels of the same category. Main results: Experimental results on the BraTS 2019 datasets show that the proposed method outperformers the state-of-the-art algorithms in both segmentation performance and computational complexity. Significance: The CGA U-Net can effectively capture the global semantic information in the MRI image by using the SAM module, while significantly reducing the computational cost. Code is available at https://github.com/delugewalker/CGA-U-Net.
BibTeX
@misc{li2022category,
  author = {Jiangyun Li and Hong Yu and Chen Chen and Meng Ding and Sen Zha},
  title = {{Category Guided Attention Network for Brain Tumor Segmentation in MRI}},
  year = {2022},
  eprint = {2203.15383},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2203.15383},
}
Figure