XNet: Wavelet-Based Low and High Frequency Fusion Networks for Fully- and Semi-Supervised Semantic Segmentation of Biomedical Images

Yanfeng Zhou, Jiaxing Huang, Chenlong Wang, Le Song, Ge Yang

Abstract

Semantic segmentation is critical for biomedical applications but is often hampered by the scarcity of expert-labeled data and the failure of standard architectures to explicitly model image frequency information. We introduce XNet, an "X-shaped" network that utilizes Discrete Wavelet Transform (DWT) to decompose images into low-frequency (LF) and high-frequency (HF) components. By processing these via specialized dual encoders and a central fusion module, XNet achieves state-of-the-art performance. Furthermore, XNet leverages frequency-based "natural perturbations" for semi-supervised learning through Cross Pseudo Supervision (CPS), reducing reliance on artificial data augmentation.

Introduction

Biomedical semantic segmentation—the pixel-wise labeling of anatomical structures—is essential for tasks like tumor detection and organ mapping. Two primary challenges persist in this field: the high cost of expert annotations and the "frequency neglect" inherent in standard convolutional neural networks (CNNs). Most models focus primarily on spatial features, ignoring the fact that LF components typically capture global semantics and shapes, while HF components capture fine edges and textures. XNet addresses these gaps by integrating frequency decomposition into a unified supervised and semi-supervised framework.

Methodology

Frequency Decomposition via DWT

The core innovation of XNet is the integration of the Discrete Wavelet Transform (DWT). Unlike standard convolutions, DWT mathematically decomposes an input image into an approximation (LF) and several detail components (HF).

Discrete Wavelet Transform (DWT) decomposition into LF and HF components.

As illustrated in the figure above, the LF image captures the essential semantic structure, while the HF image highlights boundaries and textures. This allows the model to extract specialized features for global structure and local detail independently.

The XNet Architecture

The architecture follows a distinct "X-shape" designed to exploit these frequency differences:

  1. Dual Encoders: Separate encoders extract specialized features for LF and HF components.
  2. Fusion Module: A central module integrates these features to create a comprehensive representation.
  3. Dual-Branch Decoder: The network outputs two separate segmentation predictions, facilitating consistency learning.
The XNet architecture featuring dual encoders and a fusion module.

Semi-Supervised Learning

In semi-supervised scenarios, XNet utilizes Cross Pseudo Supervision (CPS). The prediction from the LF-heavy branch serves as a pseudo-label for the HF-heavy branch, and vice versa. This "natural perturbation" strategy ensures the model remains consistent across frequency variations without needing synthetic noise or artificial augmentations that can introduce bias.

Experimental Evaluation

XNet was evaluated across four diverse biomedical benchmarks covering different modalities:

  • 2D Datasets: GlaS (histology) and CREMI (electron microscopy).
  • 3D Datasets: LA (MRI) and LiTS (CT).

Key Results

XNet consistently outperformed state-of-the-art models, including UNet++, nnUNet, and various Transformer-based architectures.

  • Supervised Learning: On the GlaS dataset, XNet improved the Jaccard Index (JI) by 2.09% and the Dice Coefficient (DC) by 1.24% over previous benchmarks.
  • Semi-Supervised Efficiency: When trained with only 20% labeled data, XNet significantly outperformed existing methods like URPC, demonstrating superior data efficiency in resource-constrained settings.
  • Model Efficiency: A lightweight variant of XNet outperformed standard UNet models while requiring significantly fewer parameters and computational resources.

Conclusion

XNet provides a robust framework for biomedical segmentation by explicitly modeling image frequencies. While it excels in detail-rich images, its performance on blurry datasets (such as ISIC-2017 skin lesions) suggests that adaptive frequency weighting is a promising area for future research. Future iterations may focus on making the frequency fusion more adaptive to input quality or exploring more complex wavelet bases.

Summary updated

Abstract

Fully- and semi-supervised semantic segmentation of biomedical images have been advanced with the development of deep neural networks (DNNs). So far, however, DNN models are usually designed to support one of these two learning schemes, unified models that support both fully- and semi-supervised segmentation remain limited. Furthermore, few fully-supervised models focus on the intrinsic low frequency (LF) and high frequency (HF) information of images to improve performance. Perturbations in consistency-based semi-supervised models are often artificially designed. They may introduce negative learning bias that are not beneficial for training. In this study, we propose a wavelet-based LF and HF fusion model XNet, which supports both fully- and semi-supervised semantic segmentation and outperforms state-of-the-art models in both fields. It emphasizes extracting LF and HF information for consistency training to alleviate the learning bias caused by artificial perturbations. Extensive experiments on two 2D and two 3D datasets demonstrate the effectiveness of our model. Code is available at https://github.com/Yanfeng-Zhou/XNet.
BibTeX
@inproceedings{zhou2023xnet,
  author = {Yanfeng Zhou and Jiaxing Huang and Chenlong Wang and Le Song and Ge Yang},
  title = {{XNet: Wavelet-Based Low and High Frequency Fusion Networks for Fully- and Semi-Supervised Semantic Segmentation of Biomedical Images}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  doi = {10.1109/iccv51070.2023.01928},
}
Figure