Shape-Adapting Gated Experts: Dynamic Expert Routing for Colonoscopic Lesion Segmentation

Gia Huy Thai, Hoang-Nguyen Vu, Anh-Minh Phan, Quang-Thinh Ly, Tram Dinh, Thi-Ngoc-Truc Nguyen, Nhat HoarXiv 2025 · DOI · arXiv

Abstract

Accurate segmentation of colonoscopic lesions is critical for cancer diagnosis but remains challenging due to the high variability in tissue morphology. Traditional static neural networks often fail to adapt to this diversity efficiently. We present Shape-Adapting Gated Experts (SAGE), a dynamic framework that employs a hierarchical Mixture-of-Experts (MoE) approach to route image features to specialized CNN or Transformer modules. SAGE achieves state-of-the-art performance on major colorectal histopathology benchmarks, demonstrating superior robustness and precision compared to fixed-architecture models.

Introduction

In digital pathology, identifying malignant tissue within gigapixel Whole Slide Images (WSIs) is a complex task. Most existing models utilize a static computation graph, processing every image patch—regardless of its structural complexity—through the same sequence of operations. This leads to computational inefficiency in simple regions and potential under-modeling in heterogeneous tumor areas. While Convolutional Neural Networks (CNNs) excel at capturing local textures and Vision Transformers (ViTs) are adept at modeling long-range spatial relationships, rigid hybrid architectures often fail to optimally leverage these complementary strengths.

Methodology

The SAGE framework introduces a dynamic, input-adaptive architecture characterized by three primary components:

  1. Dual-Path Architecture: Each layer integrates a "Main Path" for stability and an "Expert Path" for specialized computation. A learned scalar gate determines the degree of feature enrichment from the experts.
  2. Hierarchical Top-K Routing: A two-level gating system first distinguishes between shared (general-purpose) and fine-grained (specialized) experts. Subsequently, Semantic Affinity Routing (SAR) matches input queries to expert keys to select the most relevant sub-modules.
  3. Shape-Adapting Hub (SA-Hub): To bridge the architectural gap between CNNs and Transformers, the SA-Hub serves as a universal interface, reshaping and projecting data to allow seamless inter-module communication.

The implementation utilizes Sigmoid gating instead of Softmax, enabling the simultaneous activation of multiple experts based on individual merit, which enhances training stability.

Experimental Setup

The model was evaluated on three colorectal histopathology benchmarks:

  • EBHI (Extended Biopsy Histopathological Image): 795 images from the Adenocarcinoma subset.
  • DigestPath: Approximately 32,000 tissue patches extracted from 660 gigapixel WSIs.
  • GlaS (Gland Segmentation): 165 images from the MICCAI challenge, focusing on boundary precision.

Results and Discussion

SAGE-UNet established new state-of-the-art benchmarks, achieving a 95.57% Dice Score on EBHI and a 95.16% Dice Score on DigestPath (a 3.34% improvement over the ConvNeXt-UNet baseline). On the GlaS dataset, it reached 94.17% Dice Score, showing particular strength in "Test B," which contains highly irregular morphologies.

Grad-CAM visualization of the CNN Main Path showing localized attention on structural features.

Visualization via Grad-CAM confirms that SAGE dynamically redistributes its attention. As shown in the figures, the CNN experts focus on low-level structural features such as edges and textures, while the Transformer experts engage in high-level semantic reasoning across broader regions.

Grad-CAM visualization of the Transformer Main Path showing broader semantic focus.

Qualitatively, SAGE produces cleaner segmentation masks with significantly fewer false positives in the background stroma compared to models like TransUNet or Swin-UNet.

Conclusion

SAGE demonstrates that dynamic routing is a powerful paradigm for medical image segmentation. By adapting its computation to the local complexity of the tissue, it achieves superior accuracy and robustness. Future research will focus on scaling this approach to large-scale foundation models and extending its application to 3D medical imaging modalities.

Reference: Thai, G. H., Vu, H. N., et al. (2025). Shape-Adapting Gated Experts: Dynamic Expert Routing for Colonoscopic Lesion Segmentation. arXiv:2511.18493v2.

Summary updated

BibTeX
@misc{thai2025shape,
  author = {Gia Huy Thai and Hoang-Nguyen Vu and Anh-Minh Phan and Quang-Thinh Ly and Tram Dinh and Thi-Ngoc-Truc Nguyen and Nhat Ho},
  title = {{Shape-Adapting Gated Experts: Dynamic Expert Routing for Colonoscopic Lesion Segmentation}},
  year = {2025},
  eprint = {2511.18493},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.18493},
}
Figure