In the world of medical imaging, high-quality labeled data is a luxury. Semi-supervised learning (SSL) aims to bridge this gap by using a small set of labeled images alongside a larger pool of unlabeled ones. While the original XNet introduced a clever way to use wavelet transforms for this task, it had a "blind spot": it struggled when images lacked high-frequency details (like edges and textures).
XNet v2 is a significant upgrade that fixes these "frequency-blind" limitations, offering a more universal and robust architecture for both 2D and 3D biomedical image segmentation.
TL;DR
XNet v2 improves upon its predecessor by introducing image-level complementary fusion and multi-level feature fusion. By intelligently mixing low- and high-frequency information, it prevents model degradation in "blurry" datasets and achieves state-of-the-art results in semi-supervised medical segmentation.
What Problem Does This Paper Address?
The original XNet architecture split images into Low-Frequency (LF) and High-Frequency (HF) components using wavelets, processing them in separate branches. However, the authors identified three critical flaws:
- HF Dependency: If a dataset (like skin lesions) lacks sharp edges or rich textures, the HF branch fails to extract useful features, causing the whole model to underperform.
- Raw Image Neglect: XNet ignored the original raw image, missing out on holistic information that the wavelet split might have obscured.
- Insufficient Fusion: Information was only shared between branches at the deepest layers, missing opportunities to align features at shallower (edge-based) levels.
Methods
XNet v2 employs a triple-network architecture consisting of a Main network (M), an LF network (L), and an HF network (H). The core innovations include:
- Image-Level Complementary Fusion: Instead of feeding pure LF and HF images to the sub-networks, the authors "mix" them. The input to the LF branch becomes $LF + \alpha HF$, and the HF branch gets $HF + \beta LF$. This ensures that even if the HF signal is nearly zero, the branch still receives a "perturbed" version of the image to process, maintaining the stability of the consistency loss.
- Triple-Branch Consistency: The model uses Cross Pseudo Supervision (CPS). Each branch generates a prediction; the predictions from the LF and HF branches serve as "pseudo-labels" to supervise the Main branch, and vice-versa.
- Asymmetric Feature Fusion: The authors introduced modules to fuse features between the branches. Interestingly, they use shallow features for the HF-Main fusion (to capture textures) and deep features for the LF-Main fusion (to capture global semantics).
- Raw Image Integration: The Main network (M) now processes the raw image directly, ensuring no original information is lost.
Datasets
The authors evaluated XNet v2 on a diverse range of 2D and 3D public biomedical datasets:
- 2D Datasets: GlaS (Gland segmentation), CREMI (Electron microscopy), and ISIC-2017 (Skin lesion dermoscopy).
- 3D Datasets: P-CT (Pancreas CT) and LiTS (Liver Tumor Segmentation).
These datasets represent a wide variety of imaging modalities and frequency characteristics, providing a rigorous test for "universality."
Key Insights and Ideas
- Wavelets as Natural Perturbations: In SSL, we often add random noise to images to force the model to be consistent. XNet v2 treats frequency decomposition as a more "principled" form of perturbation. By slightly mixing the frequencies, they create a more meaningful challenge for the model than simple Gaussian noise.
- Solving the "HF Silence" Problem: The "aha moment" is the realization that the HF branch shouldn't just be for high frequencies. By adding a bit of LF information to the HF branch, they prevent the branch from "dying" when the input image is smooth, effectively turning the HF branch into a specialized edge-enhancer that still understands the base shape.
- Efficiency via Pruning: Despite having three branches, the authors show that you can halve or even quarter the number of channels (XNet v2- and v2--) and still outperform much larger, standard models like MT (Mean Teacher) or CCT.
Key Results
XNet v2 consistently outperformed previous state-of-the-art (SOTA) semi-supervised models across all benchmarks:
- ISIC-2017 (The "Hard" Dataset): On the skin lesion dataset where the original XNet struggled, XNet v2 improved the Jaccard score from 71.17% to 74.07%, proving its improved universality.
- 3D Performance: On the P-CT dataset, XNet 3D v2 achieved a Dice score of 80.21%, significantly higher than the original XNet 3D's 75.67%.
- Fully Supervised: Even when 100% of labels are available, XNet v2 remains highly competitive, often beating standard UNet and the original XNet.
Limitations and Future Work
While XNet v2 is more robust, it still relies on the wavelet transform as a fixed preprocessing step. The authors suggest that exploring learnable frequency decompositions could be a next step. Additionally, while they provide lightweight versions, the triple-branch architecture inherently involves more parameters during the training phase than a single-stream model, even if only the Main branch is used during inference.
The code is available at: https://github.com/Yanfeng-Zhou/XNetv2
Summary updated