TL;DR
TopoMamba is a new framework for medical image segmentation that enhances Visual State-Space Models (SSMs) by introducing diagonal scanning patterns to better capture curved and oblique anatomical structures. By combining these "topology-aware" scans with a lightweight, dependence-based fusion gate and a smart index-caching mechanism, the authors achieve state-of-the-art performance across CT, dermoscopy, and endoscopy images while significantly reducing inference latency.
What Problem Does This Paper Address?
Medical images rarely follow a perfect grid. Anatomical structures like the pancreas or gallbladder are often thin, curved, or oriented at odd angles. However, most modern Visual State-Space Models (SSMs)—which are popular for their linear computational efficiency compared to Transformers—rely on "Cross-Scan" patterns that traverse images only horizontally and vertically.
This axis-biased approach often "breaks" the topological connectivity of non-axial structures, leading to fragmented segmentations. Furthermore, when researchers try to fix this by adding more scanning branches, they often use naive fusion methods (like simple addition) that amplify redundant information. Finally, recomputing the complex pixel-traversal indices for every image during inference creates a significant bottleneck in real-world clinical deployment.
Methods
The authors propose TopoMamba, a U-shaped architecture built on three core innovations:
- TopoA-Scan (Topology-Aware Scan): In addition to standard horizontal/vertical scans, this module traverses the feature grid along alternating diagonals and anti-diagonals. This provides a structural bias that is much better suited for capturing the "complex-curve continuity" found in medical media.
- HSIC Gate: To merge the standard Cross-Scan features with the new TopoA-Scan features, the authors designed a gate based on the Hilbert-Schmidt Independence Criterion (HSIC). Instead of a heavy fusion head, it uses a single learnable scalar to adaptively balance the branches based on their statistical dependence. This prevents the model from being overwhelmed by redundant data.
- ScanCache: Recognizing that clinical workflows often use fixed resolutions, the authors introduced a device-aware caching mechanism. It stores the scan indices in GPU memory after the first traversal, converting an $O(HW)$ construction task into a simple, fast lookup for all subsequent frames.
The framework is instantiated in both 2D and a volumetric TopoMamba-3D version for practical clinical use.
Datasets
The model was evaluated on three heterogeneous benchmarks to test its generalizability:
- Synapse Multi-organ CT: Challenging 3D abdominal anatomy with weak organ boundaries.
- ISIC 2017 Dermoscopy: Skin lesion images featuring irregular contours and distractors like hair.
- CVC-ClinicDB Endoscopy: Polyp segmentation in video frames with motion blur and specular highlights.
Key Insights and Ideas
- Diagonal Bias Matters: The "aha moment" here is that diagonal scanning isn't just an extra branch; it's a complementary structural prior. While Cross-Scan handles axis-aligned boundaries, TopoA-Scan excels at preserving the continuity of thin, oblique organs.
- Fusion via Independence: Most fusion methods try to learn what to combine. The HSIC Gate effectively asks how much the branches overlap statistically, using that measure to regulate their interaction. This lightweight approach proves more stable than complex fusion heads.
- Deployment as a First-Class Citizen: By including ScanCache, the authors acknowledge that algorithmic elegance doesn't matter if the overhead of reordering pixels makes the model too slow for a surgical suite.
Key Results
TopoMamba consistently outperformed strong CNN, Transformer, and SSM baselines:
- Accuracy: On the Synapse CT dataset, TopoMamba-3D achieved a 91.28% average Dice score, notably boosting performance on difficult organs like the pancreas (improving Dice from ~60% in standard SSMs to over 81%).
- Efficiency: ScanCache provided a massive speedup. For $512 \times 512$ inputs, it reduced latency by 47.9% (from 72.1ms down to 37.6ms).
- Qualitative Quality: Visual results show that TopoMamba produces much cleaner, more connected masks for curved structures compared to models like Mamba-UNet or Swin-UMamba, which often produce "broken" or noisy regions.
Limitations and Future Work
The authors note that while TopoMamba is "topology-aware" (it uses structural biases to improve results), it is not "topology-guaranteeing." It does not use hard mathematical constraints to ensure, for example, that a segmented tube has no holes.
Future work could involve integrating explicit topological losses or exploring how these diagonal scanning patterns might benefit other domains beyond medical imaging, such as remote sensing or industrial defect detection. Not discussed in the paper is the model's performance on extremely low-resolution or highly compressed medical video streams.
Summary updated