Spatial-Frequency Dual Progressive Attention Network For Medical Image Segmentation

Zhenhuan Zhou, Along He, Yanlin Wu, Rui Yao, Xueshuo Xie, Tao Li

Abstract

Medical image segmentation is a critical task complicated by significant variations in lesion size, shape, and texture. Traditional Convolutional Neural Networks (CNNs) often struggle with global context, while Transformer-based models can overlook fine-grained high-frequency details. This paper introduces SF-UNet, a dual-domain architecture that integrates spatial and frequency information. By employing a Multi-scale Progressive Channel Attention (MPCA) block for cross-scale feature fusion and a lightweight Frequency-Spatial Attention (FSA) block for spectral domain processing, the network captures both global structures and sharp boundary details. Experimental results on ISIC-2018, BUSI, and NKUT datasets demonstrate that SF-UNet achieves state-of-the-art performance while maintaining high computational efficiency.

Introduction

Accurate segmentation of medical images, such as skin lesions or tumors, is hindered by two primary limitations in existing architectures. First, the Receptive Field Gap in standard CNNs limits their ability to model long-range dependencies due to small kernel sizes. Second, the Frequency Bias in global modeling techniques (like Transformers) often leads to the smoothing of high-frequency information, which is essential for defining precise boundaries and intricate textures.

To address these issues, we propose SF-UNet. The architecture is designed to "see the forest and the trees" by explicitly processing images in both the spatial domain (pixels) and the frequency domain (spectral patterns). This dual-domain approach allows the model to leverage global context without the heavy computational overhead of standard self-attention mechanisms.

Methodology

SF-UNet enhances the traditional U-Net framework with two innovative modules designed to bridge the gap between local and global features.

Overview of the SF-UNet architecture, featuring the Multi-scale Progressive Channel Attention (MPCA) and Frequency-Spatial Attention (FSA) modules.

Multi-scale Progressive Channel Attention (MPCA)

Instead of treating encoder layers independently, the MPCA block facilitates "cross-scale" communication by fusing features from adjacent layers. This progressive fusion helps the network adapt to lesions of varying sizes and reduces feature redundancy across the encoder hierarchy.

Frequency-Spatial Attention (FSA)

The FSA block serves as a lightweight mechanism (adding only 0.05M parameters) to refine features in both domains:

  • Spatial Branch: Utilizes standard attention to highlight salient pixel regions.
  • Frequency Branch: Employs a 2D Discrete Fourier Transform (DFT) to map features into the spectral domain. It separates data into low-frequency components (representing global structure) and high-frequency components (representing edges). A learnable filter adaptively weights these components before they are transformed back to the spatial domain via inverse DFT.

Experiments and Results

The model was evaluated across three diverse medical imaging datasets: ISIC-2018 (skin lesions), BUSI (breast ultrasound), and NKUT (pediatric dental CBCT).

Quantitative Performance

SF-UNet consistently outperformed previous state-of-the-art (SOTA) models. On the ISIC-2018 dataset, it achieved a Dice Similarity Coefficient (DSC) of 88.46%, surpassing high-capacity models like TransUNet (92.23M parameters) despite having a much smaller footprint of 28.89M parameters.

Quantitative comparison of SF-UNet against state-of-the-art models on the ISIC-2018 dataset.

Key Insights

  • Efficiency of the Fourier Domain: Using the frequency domain acts as a "global shortcut." Global information is concentrated in low-frequency components, allowing for effective global learning at a fraction of the cost of Transformer-based self-attention.
  • Boundary Refinement: By explicitly preserving high-frequency components, SF-UNet avoids the blurring effects common in deep learning, resulting in sharper segmentation boundaries. This was particularly evident in the BUSI dataset, where SF-UNet significantly reduced the Hausdorff Distance (HD95).

Conclusion

SF-UNet represents a significant step forward in medical image segmentation by effectively combining spatial and frequency domain learning. The synergy between MPCA and FSA allows the model to handle multi-scale localization and texture refinement simultaneously. While pure Transformer models like TransUNet may still hold a slight edge in specific high-capacity scenarios (e.g., certain dental structures in the NKUT dataset), SF-UNet offers a more balanced and efficient solution for general medical diagnostic tasks. Future work will explore optimizing learnable frequency filters and extending the architecture to native 3D volumetric data.

Summary updated

BibTeX
@inproceedings{zhou2024spatial,
  author = {Zhenhuan Zhou and Along He and Yanlin Wu and Rui Yao and Xueshuo Xie and Tao Li},
  title = {{Spatial-Frequency Dual Progressive Attention Network For Medical Image Segmentation}},
  booktitle = {BIBM},
  year = {2024},
  doi = {10.1109/bibm62325.2024.10822613},
}
Figure