TopoMamba: Topology-Aware Scanning and Fusion for Segmenting Heterogeneous Medical Visual Media

Fuchen Zheng, Chengpei Xu, Long Ma, Weixuan Li, Junhua Zhou, Xuhang Chen, Weihuang Liu, Haolun Li, Quanjun Li, Zhenxi Zhang, Lei Zhao, Chi-Man Pun, Shoujun ZhouarXiv 2026 · arXiv

TL;DR

TopoMamba is a new framework for medical image segmentation that enhances Visual State-Space Models (SSMs) by introducing diagonal scanning patterns to better capture curved and oblique anatomical structures. By combining these "topology-aware" scans with a lightweight, dependence-based fusion gate and a smart index-caching mechanism, the authors achieve state-of-the-art performance across CT, dermoscopy, and endoscopy images while significantly reducing inference latency.

What Problem Does This Paper Address?

Medical images rarely follow a perfect grid. Anatomical structures like the pancreas or gallbladder are often thin, curved, or oriented at odd angles. However, most modern Visual State-Space Models (SSMs)—which are popular for their linear computational efficiency compared to Transformers—rely on "Cross-Scan" patterns that traverse images only horizontally and vertically.

This axis-biased approach often "breaks" the topological connectivity of non-axial structures, leading to fragmented segmentations. Furthermore, when researchers try to fix this by adding more scanning branches, they often use naive fusion methods (like simple addition) that amplify redundant information. Finally, recomputing the complex pixel-traversal indices for every image during inference creates a significant bottleneck in real-world clinical deployment.

Fig. 1. Motivation and overview of TopoMamba. Left: axis-biased scanning disrupts non-axial structural continuity in 3D CT, endoscopy, and dermoscopy. Right: Cross-Scan and TopoA-Scan are fused by the HSIC Gate to better preserve non-axial continuity and suppress false positives.

Methods

The authors propose TopoMamba, a U-shaped architecture built on three core innovations:

  1. TopoA-Scan (Topology-Aware Scan): In addition to standard horizontal/vertical scans, this module traverses the feature grid along alternating diagonals and anti-diagonals. This provides a structural bias that is much better suited for capturing the "complex-curve continuity" found in medical media.
  2. HSIC Gate: To merge the standard Cross-Scan features with the new TopoA-Scan features, the authors designed a gate based on the Hilbert-Schmidt Independence Criterion (HSIC). Instead of a heavy fusion head, it uses a single learnable scalar to adaptively balance the branches based on their statistical dependence. This prevents the model from being overwhelmed by redundant data.
  3. ScanCache: Recognizing that clinical workflows often use fixed resolutions, the authors introduced a device-aware caching mechanism. It stores the scan indices in GPU memory after the first traversal, converting an $O(HW)$ construction task into a simple, fast lookup for all subsequent frames.

The framework is instantiated in both 2D and a volumetric TopoMamba-3D version for practical clinical use.

Architecture of TopoMamba-3D showing the ETMS block and HSIC Gate

Datasets

The model was evaluated on three heterogeneous benchmarks to test its generalizability:

  • Synapse Multi-organ CT: Challenging 3D abdominal anatomy with weak organ boundaries.
  • ISIC 2017 Dermoscopy: Skin lesion images featuring irregular contours and distractors like hair.
  • CVC-ClinicDB Endoscopy: Polyp segmentation in video frames with motion blur and specular highlights.

Key Insights and Ideas

  • Diagonal Bias Matters: The "aha moment" here is that diagonal scanning isn't just an extra branch; it's a complementary structural prior. While Cross-Scan handles axis-aligned boundaries, TopoA-Scan excels at preserving the continuity of thin, oblique organs.
  • Fusion via Independence: Most fusion methods try to learn what to combine. The HSIC Gate effectively asks how much the branches overlap statistically, using that measure to regulate their interaction. This lightweight approach proves more stable than complex fusion heads.
  • Deployment as a First-Class Citizen: By including ScanCache, the authors acknowledge that algorithmic elegance doesn't matter if the overhead of reordering pixels makes the model too slow for a surgical suite.

Key Results

TopoMamba consistently outperformed strong CNN, Transformer, and SSM baselines:

  • Accuracy: On the Synapse CT dataset, TopoMamba-3D achieved a 91.28% average Dice score, notably boosting performance on difficult organs like the pancreas (improving Dice from ~60% in standard SSMs to over 81%).
  • Efficiency: ScanCache provided a massive speedup. For $512 \times 512$ inputs, it reduced latency by 47.9% (from 72.1ms down to 37.6ms).
  • Qualitative Quality: Visual results show that TopoMamba produces much cleaner, more connected masks for curved structures compared to models like Mamba-UNet or Swin-UMamba, which often produce "broken" or noisy regions.
Qualitative comparison on ISIC and CVC-ClinicDB datasets

Limitations and Future Work

The authors note that while TopoMamba is "topology-aware" (it uses structural biases to improve results), it is not "topology-guaranteeing." It does not use hard mathematical constraints to ensure, for example, that a segmented tube has no holes.

Future work could involve integrating explicit topological losses or exploring how these diagonal scanning patterns might benefit other domains beyond medical imaging, such as remote sensing or industrial defect detection. Not discussed in the paper is the model's performance on extremely low-resolution or highly compressed medical video streams.

Summary updated

Abstract

Visual state-space models (SSMs) have shown strong potential for medical image segmentation, yet their effectiveness is often limited by two practical issues: axis-biased scan ordering weakens the modeling of oblique and curved structures, and naive multi-branch fusion tends to amplify redundant responses. We present TopoMamba, a topology-aware scan-and-fuse framework for segmenting heterogeneous medical visual media. The method combines a diagonal/anti-diagonal TopoA-Scan branch with the standard Cross-Scan branch to provide complementary structural priors, and introduces ScanCache, a device-aware caching mechanism that amortizes explicit scan-index construction across recurring resolutions. To fuse heterogeneous scan features efficiently, we further propose a lightweight HSIC Gate that regulates branch interaction using a dependence-aware scalar gating rule. We also instantiate a volumetric TopoMamba-3D for practical 3D clinical segmentation. Experiments on Synapse CT, ISIC 2017 dermoscopy, and CVC-ClinicDB endoscopy show that TopoMamba consistently improves segmentation quality over strong CNN, Transformer, and SSM baselines, with particularly clear gains on thin or curved targets such as the pancreas and gallbladder, while maintaining favorable deployment efficiency under dynamic input resolutions. These results suggest that topology-aware scan ordering and lightweight dependence-aware fusion form an effective and practical design for medical multimedia segmentation. The code will be made publicly available.
BibTeX
@misc{zheng2026topomamba,
  author = {Fuchen Zheng and Chengpei Xu and Long Ma and Weixuan Li and Junhua Zhou and Xuhang Chen and Weihuang Liu and Haolun Li and Quanjun Li and Zhenxi Zhang and Lei Zhao and Chi-Man Pun and Shoujun Zhou},
  title = {{TopoMamba: Topology-Aware Scanning and Fusion for Segmenting Heterogeneous Medical Visual Media}},
  year = {2026},
  eprint = {2604.25545},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2604.25545},
}

Loading related content...

Figure