FAMNet: Frequency-aware Matching Network for Cross-domain Few-shot Medical Image Segmentation

Introduction

Few-shot medical image segmentation (FSMIS) is a critical task aimed at segmenting novel organs or lesions using only a minimal number of annotated examples. A significant challenge in this field is domain shift, where models trained on one modality (e.g., CT) perform poorly on another (e.g., MRI) due to differences in intensity distributions and imaging styles. Existing spatial-domain methods often struggle to decouple semantic content from these domain-specific styles.

Motivation: Frequency over Space

The authors identify two primary hurdles: intra-domain variance (patient-specific differences) and inter-domain variance (modality-specific differences). While spatial features are highly sensitive to these shifts, frequency-domain characteristics—specifically mid-frequency components—remain remarkably stable across different imaging modalities.

Expand

Source, p. 1

As shown in Figure 1, while it is difficult to decouple semantics and styles in the spatial domain, frequency bands are more easily separated. Similarity metrics like SSIM are significantly higher in the frequency domain for mid-range bands, suggesting that these components are more "domain-agnostic."

The FAMNet Framework

FAMNet addresses domain shift by shifting the feature-matching process to the frequency domain. The architecture consists of three integrated modules:

Expand

Source, p. 3

1. Coarse Prediction Generation (CPG): This stage uses an initial prototypical network to generate a rough mask of the target, helping to isolate foreground features and reduce support-query bias.
2. Frequency-aware Matching (FAM) Module: Utilizing the Fast Fourier Transform (FFT), this module decomposes features into low, medium, and high frequency bands. It applies Attention-based Matching (ABM) selectively, enhancing similarities in stable mid-frequency bands while suppressing noise and style-heavy information in low and high frequencies.
3. Multi-Spectral Fusion (MSF) Module: This module employs a cross-attention (CA) mechanism to refine the features. It uses the stable mid-frequency information as a guide to extract useful structural details from the other bands, resulting in a robust, final foreground prototype.

Experimental Results

FAMNet was evaluated across three challenging scenarios: cross-modality (Abdominal CT to MRI), cross-sequence (Cardiac LGE to b-SSFP), and cross-institution (Prostate MRI).

• State-of-the-Art Performance: On the Abdominal CT $\rightarrow$ MRI task, FAMNet achieved a 65.79% Dice score, outperforming previous methods by a significant margin.
• Cardiac Imaging Gains: In the LGE $\rightarrow$ b-SSFP shift, the model reached 71.58% Dice, compared to the second-best method's 61.44%.
• Ablation Insights: Studies confirmed that the frequency-specific attention is vital; applying uniform attention across all bands resulted in a performance drop of over 7%.

Conclusion and Future Work

FAMNet demonstrates that the frequency domain offers a more stable foundation for cross-domain few-shot segmentation than the spatial domain. By selectively attending to domain-agnostic frequency bands, the model achieves superior generalization. Future work may explore automated methods for frequency band decoupling and improving edge detection for small targets, where high-frequency noise remains a challenge.