Introduction
Medical image segmentation—the precise identification of structures like tumors or blood vessels—requires balancing local detail with global context. While Convolutional Neural Networks (CNNs) excel at capturing local textures, they often fail to model long-range dependencies. Conversely, Transformers provide global context but suffer from quadratic computational complexity as image resolution increases.
Recently, State Space Models like Mamba have emerged as linear-complexity alternatives to Transformers. However, existing Mamba-based models often overlook raw perceptual cues and sacrifice local precision. PGM-UNet addresses these limitations by introducing a prompt-guided dual-path architecture that integrates Mamba's global modeling with CNN-based local extraction.
Architecture and Methodology
The core of PGM-UNet is the LG-Net (Local-Global Network), integrated into a standard UNet-style encoder-decoder framework. This dual-path approach ensures that both fine-grained details and broad contextual relationships are preserved.
The architecture consists of several key innovations:
- Prompt-Guided Residual Mamba (PGRM): This module uses the original input image as a "visual prompt" to adaptively guide the Mamba module, ensuring the network retains raw structural information during deep feature abstraction.
- Local Information Extraction Module (LIEM): A parallel CNN path dedicated to capturing high-resolution local features.
- Multi-focus Attention Fusion Module (MAFM): An intelligent fusion mechanism that uses channel attention to weight and combine local and global features, effectively suppressing noise.
- KAN-based Bottleneck (MIEM): At the deepest layer, the model employs Kolmogorov-Arnold Networks (KANs) with dilated convolutions. Unlike traditional fixed-activation layers, KANs use learnable functions to extract complex non-linear features without losing spatial resolution.
Experimental Results
PGM-UNet was evaluated on five benchmarks, including skin lesion datasets (ISIC-2017, ISIC-2018, PH2) and vessel segmentation tasks (DIAS, DRIVE).
Performance and Efficiency
A defining characteristic of PGM-UNet is its extreme efficiency. With only 5.48M parameters, it outperforms significantly larger models. For instance, on the ISIC-2018 dataset, it achieved a Dice Similarity Coefficient (DSC) of 89.48%, surpassing the next best Mamba-based model by over 1%.
In vessel segmentation (DIAS), the model reached a DSC of 75.31%, a substantial improvement over the 65.07% achieved by the original UNet. Furthermore, the model demonstrated strong generalization; when trained on ISIC and tested on the unseen PH2 dataset, it maintained a high DSC of 92.01%.
Conclusion and Future Work
PGM-UNet represents a significant step forward in efficient medical image segmentation by successfully bridging the gap between local CNN features and global Mamba modeling. While the model shows state-of-the-art performance, challenges remain in handling heavy occlusions (e.g., hair in skin images) and maintaining connectivity in extremely dense micro-vessel networks. Future research will focus on expanding this prompt-guided approach into a general-purpose foundation model for diverse medical imaging modalities.
Summary updated