Leveraging Frequency Domain Learning in 3D Vessel Segmentation

Xinyuan Wang, Chengwei Pan, Hongming Dai, Gangming Zhao, Jinpeng Li, Xiao Zhang, Yizhou Yu

Abstract

Coronary microvascular disease constitutes a substantial risk to human health. Employing computer-aided analysis and diagnostic systems, medical professionals can intervene early in disease progression, with 3D vessel segmentation serving as a crucial component. Nevertheless, conventional U-Net architectures tend to yield incoherent and imprecise segmentation outcomes, particularly for small vessel structures. While models with attention mechanisms, such as Transformers and large convolutional kernels, demonstrate superior performance, their extensive computational demands during training and inference lead to increased time complexity. In this study, we leverage Fourier domain learning as a substitute for multi-scale convolutional kernels in 3D hierarchical segmentation models, which can reduce computational expenses while preserving global receptive fields within the network. Furthermore, a zero-parameter frequency domain fusion method is designed to improve the skip connections in U-Net architecture. Experimental results on a public dataset and an in-house dataset indicate that our novel Fourier transformation-based network achieves remarkable dice performance (84.37% on ASACA500 and 80.32% on ImageCAS) in tubular vessel segmentation tasks and substantially reduces computational requirements without compromising global receptive fields.

Introduction

Segmenting coronary arteries from 3D CT scans is a high-stakes task for diagnosing heart disease. However, vessels are notoriously difficult to map because they are thin, branching, and span long distances across an image. Standard convolutional networks, such as the U-Net, often struggle because their receptive fields are too local, making it difficult to capture the continuity of long, winding vessels. While Transformer-based models address this by using self-attention to capture long-range dependencies, they are computationally expensive in 3D environments, requiring significant memory and processing power. Additionally, Transformers can smooth out high-frequency details, such as sharp vessel walls, which are critical for accurate clinical analysis.

Methodology

To address these challenges, we propose Fseg (Fourier Segmentation), a hierarchical architecture that replaces computationally intensive self-attention with efficient Fourier domain learning. The framework operates on two primary innovations:

  1. The Fourier Block: Instead of using self-attention to identify global patterns, Fseg transforms features into the frequency domain using a Discrete Fourier Transform (DFT). In this space, global operations are performed via simple element-wise multiplication with learnable weights. This provides a global receptive field with $O(N \log N)$ complexity, a significant improvement over the $O(N^2)$ complexity of standard attention mechanisms.
  2. Fourier Fusion Decoder: Traditional skip connections in U-Nets typically use simple concatenation. Fseg introduces a zero-parameter frequency-aware fusion strategy. This method uses filters to isolate low-frequency semantic information (general organ shape) and high-frequency structural details (fine vessel edges) from different layers, merging them to produce a more precise segmentation map.
  3. Anti-Aliasing Padding: To ensure spatial integrity, a specialized padding strategy is implemented to mitigate "wrap-around" artifacts, a common issue when performing convolutions in the frequency domain.
The Fseg architecture overview, illustrating the hierarchical structure and the internal components of the Fourier Block.

Experiments and Results

The model was evaluated on two major cardiac datasets: ImageCAS, a public dataset of 1,000 3D CTA images, and ASACA (Automatic Segmentation of Aorta and Coronary Arteries).

Quantitative Performance

The Fseg-L (Large) model established new benchmarks for coronary vessel segmentation, achieving a 84.37% Dice score on ASACA500 and 80.32% on ImageCAS. These results outperform established models like 3D UXNet and SwinUNETR. Furthermore, Fseg demonstrates exceptional efficiency; the smallest variant, Fseg-S, utilizes only ~40G FLOPs while outperforming the standard U-Net, which requires 135G FLOPs.

Qualitative Analysis

Visual comparisons highlight Fseg's ability to maintain vessel continuity. While baseline models often produce "broken" vessel structures or noisy blobs in smaller branches, Fseg generates smoother and more continuous vessel strings, which is vital for accurate clinical diagnosis.

Qualitative comparison of segmentation results across different models. Fseg (Ours) demonstrates superior continuity in small vessel branches compared to U-Net, SwinUNETR, and UXNET.

Conclusion

Fseg offers a powerful and efficient alternative to attention-based 3D segmentation models. By leveraging the Fourier domain, the network achieves a global receptive field and superior detail preservation with significantly lower computational overhead. Future research will focus on adapting this frequency-domain approach to other complex tubular structures and exploring its flexibility across varying input resolutions.

Summary updated

Abstract

Coronary microvascular disease constitutes a substantial risk to human health. Employing computer-aided analysis and diagnostic systems, medical professionals can intervene early in disease progression, with 3D vessel segmentation serving as a crucial component. Nevertheless, conventional U-Net architectures tend to yield incoherent and imprecise segmentation outcomes, particularly for small vessel structures. While models with attention mechanisms, such as Transformers and large convolutional kernels, demonstrate superior performance, their extensive computational demands during training and inference lead to increased time complexity. In this study, we leverage Fourier domain learning as a substitute for multi-scale convolutional kernels in 3D hierarchical segmentation models, which can reduce computational expenses while preserving global receptive fields within the network. Furthermore, a zero-parameter frequency domain fusion method is designed to improve the skip connections in U-Net architecture. Experimental results on a public dataset and an in-house dataset indicate that our novel Fourier transformation-based network achieves remarkable dice performance (84.37\% on ASACA500 and 80.32\% on ImageCAS) in tubular vessel segmentation tasks and substantially reduces computational requirements without compromising global receptive fields.
BibTeX
@inproceedings{wang2023leveraging,
  author = {Xinyuan Wang and Chengwei Pan and Hongming Dai and Gangming Zhao and Jinpeng Li and Xiao Zhang and Yizhou Yu},
  title = {{Leveraging Frequency Domain Learning in 3D Vessel Segmentation}},
  booktitle = {BIBM},
  year = {2023},
  url = {https://openreview.net/forum?id=AfnufAd6P8Y},
}
Figure