Abstract
Accurate segmentation of colonoscopic lesions is critical for cancer diagnosis but remains challenging due to the high variability in tissue morphology. Traditional static neural networks often fail to adapt to this diversity efficiently. We present Shape-Adapting Gated Experts (SAGE), a dynamic framework that employs a hierarchical Mixture-of-Experts (MoE) approach to route image features to specialized CNN or Transformer modules. SAGE achieves state-of-the-art performance on major colorectal histopathology benchmarks, demonstrating superior robustness and precision compared to fixed-architecture models.
Introduction
In digital pathology, identifying malignant tissue within gigapixel Whole Slide Images (WSIs) is a complex task. Most existing models utilize a static computation graph, processing every image patch—regardless of its structural complexity—through the same sequence of operations. This leads to computational inefficiency in simple regions and potential under-modeling in heterogeneous tumor areas. While Convolutional Neural Networks (CNNs) excel at capturing local textures and Vision Transformers (ViTs) are adept at modeling long-range spatial relationships, rigid hybrid architectures often fail to optimally leverage these complementary strengths.
Methodology
The SAGE framework introduces a dynamic, input-adaptive architecture characterized by three primary components:
- Dual-Path Architecture: Each layer integrates a "Main Path" for stability and an "Expert Path" for specialized computation. A learned scalar gate determines the degree of feature enrichment from the experts.
- Hierarchical Top-K Routing: A two-level gating system first distinguishes between shared (general-purpose) and fine-grained (specialized) experts. Subsequently, Semantic Affinity Routing (SAR) matches input queries to expert keys to select the most relevant sub-modules.
- Shape-Adapting Hub (SA-Hub): To bridge the architectural gap between CNNs and Transformers, the SA-Hub serves as a universal interface, reshaping and projecting data to allow seamless inter-module communication.
The implementation utilizes Sigmoid gating instead of Softmax, enabling the simultaneous activation of multiple experts based on individual merit, which enhances training stability.
Experimental Setup
The model was evaluated on three colorectal histopathology benchmarks:
- EBHI (Extended Biopsy Histopathological Image): 795 images from the Adenocarcinoma subset.
- DigestPath: Approximately 32,000 tissue patches extracted from 660 gigapixel WSIs.
- GlaS (Gland Segmentation): 165 images from the MICCAI challenge, focusing on boundary precision.
Results and Discussion
SAGE-UNet established new state-of-the-art benchmarks, achieving a 95.57% Dice Score on EBHI and a 95.16% Dice Score on DigestPath (a 3.34% improvement over the ConvNeXt-UNet baseline). On the GlaS dataset, it reached 94.17% Dice Score, showing particular strength in "Test B," which contains highly irregular morphologies.
Visualization via Grad-CAM confirms that SAGE dynamically redistributes its attention. As shown in the figures, the CNN experts focus on low-level structural features such as edges and textures, while the Transformer experts engage in high-level semantic reasoning across broader regions.
Qualitatively, SAGE produces cleaner segmentation masks with significantly fewer false positives in the background stroma compared to models like TransUNet or Swin-UNet.
Conclusion
SAGE demonstrates that dynamic routing is a powerful paradigm for medical image segmentation. By adapting its computation to the local complexity of the tissue, it achieves superior accuracy and robustness. Future research will focus on scaling this approach to large-scale foundation models and extending its application to 3D medical imaging modalities.
Reference: Thai, G. H., Vu, H. N., et al. (2025). Shape-Adapting Gated Experts: Dynamic Expert Routing for Colonoscopic Lesion Segmentation. arXiv:2511.18493v2.
Summary updated