Abstract
Accurate polyp segmentation in colonoscopy images is a critical step in the early detection and prevention of colorectal cancer (CRC). However, existing computer vision models often struggle with the "camouflage effect," where polyps blend into surrounding healthy tissue, and "feature misalignment," where multi-scale information fusion results in blurred boundaries. This paper introduces PSTNet, a novel architecture that leverages a Shunted Transformer backbone, frequency-domain analysis via Discrete Cosine Transforms (DCT), and a specialized feature alignment module. Experimental results across five benchmark datasets—Kvasir-SEG, CVC-ClinicDB, CVC-ColonDB, ETIS, and CVC-300—demonstrate that PSTNet achieves state-of-the-art performance, particularly in challenging low-contrast and small-polyp scenarios.
Introduction
Colorectal cancer remains a leading cause of cancer-related mortality worldwide. While colonoscopies are the gold standard for identifying and removing precancerous polyps, the visual complexity of the gastrointestinal tract presents significant challenges for both clinicians and automated systems. Two primary bottlenecks hinder current segmentation models:
- The Camouflage Effect: Polyps frequently exhibit color and texture characteristics nearly identical to the surrounding mucosa, making them difficult to distinguish in the standard RGB color space.
- Feature Misalignment: Modern architectures often fuse features from different network depths to capture both global context and local detail. However, the operations of downsampling and upsampling often introduce "pixel shifts," leading to noisy feature maps and imprecise boundaries.
PSTNet addresses these issues by integrating frequency-domain cues and a dedicated alignment strategy to refine the segmentation process.
Methodology
The PSTNet architecture is composed of four primary innovative components designed to enhance feature extraction and fusion.
Shunted Transformer Encoder
Unlike standard Transformers that process tokens at a fixed scale, the Shunted Transformer backbone utilizes "shunted self-attention." This allows the model to simultaneously model objects of varying sizes, providing a more efficient way to capture the fine-grained details necessary for identifying small or irregularly shaped polyps.
Frequency Characterization Attention Module (FCAM)
To overcome the camouflage effect, the FCAM incorporates a 2D Discrete Cosine Transform (DCT). By analyzing images in the frequency domain, the model can identify subtle structural patterns and rate-of-change signatures in pixel intensity that are often invisible in the RGB space. This "hidden" cue allows the network to distinguish polyps from healthy tissue even when their colors are nearly identical.
Feature Supplementary Alignment Module (FSAM)
The FSAM is designed to resolve the problem of feature misalignment. It employs a multi-scale subtraction strategy to highlight the unique "residual" information between adjacent layers, effectively reducing redundancy. Furthermore, it utilizes Deformable Convolutions (DCNv2) to physically align features from different scales before they are fused, resulting in significantly sharper edge detection.
Cross Perception Localization Module (CPM)
The CPM serves as the final integration stage, acting as a bridge that merges frequency-rich low-level features with high-level semantic information. This ensures that the final segmentation mask is both contextually accurate and spatially precise.
Experimental Results
PSTNet was evaluated against five major benchmarks, consistently outperforming current state-of-the-art (SOTA) models such as SegT and ADSNet.
Quantitative Performance
On the Kvasir-SEG dataset, PSTNet achieved a 93.5% mDice score and an 89.5% mIoU. These metrics indicate a high degree of overlap between the predicted masks and the ground truth.
Generalization and Robustness
The model's robustness was particularly evident on the ETIS dataset, which is widely regarded as the most difficult benchmark due to the prevalence of very small polyps. PSTNet outperformed the next best model (SegT) by 1.0% in mDice, suggesting superior generalization capabilities on unseen or difficult clinical data.
Qualitative Analysis
Visual comparisons indicate that PSTNet excels at identifying tiny polyps that other models frequently overlook. Additionally, the boundaries produced by PSTNet are cleaner and more anatomically accurate, even in low-contrast scenarios where the polyp is heavily camouflaged.
Discussion and Conclusion
PSTNet represents a significant advancement in medical image segmentation by demonstrating that frequency-domain information is a powerful supplement to traditional spatial features. By combining the Shunted Transformer's efficiency with the FSAM's alignment precision, the model effectively mitigates the most common failure modes in polyp detection.
Future Work
Future research will focus on the clinical translation of PSTNet. This includes optimizing the model for real-time performance on standard hospital hardware to support live streaming colonoscopies. Additionally, the authors aim to collaborate with medical experts to validate the model's performance across more diverse, large-scale datasets containing rare polyp morphologies.
Summary updated