Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection

Dingwen Zhang, Liangbo Cheng, Yi Liu, Xinggang Wang, Junwei Han

TL;DR

MCRNet introduces a novel type-level capsule routing mechanism by leveraging the implicit latent states of Vision Mamba, drastically reducing the computational overhead of traditional pixel-level Capsule Networks. This lightweight architecture achieves state-of-the-art performance in camouflaged object detection (COD) by efficiently modeling part-whole relationships without sacrificing spatial resolution.

What Problem Does This Paper Address?

Camouflaged Object Detection (COD) aims to identify and segment targets that blend seamlessly into their surroundings. Traditional Convolutional Neural Networks (CNNs) and Transformers often struggle with this task because they look for highly discriminative regions, frequently missing fine details or failing to segment the object as a cohesive whole.

Capsule Networks (CapsNets) offer a elegant solution by modeling "part-whole" relationships (e.g., understanding how the parts of a camouflaged animal relate to its entire body). However, standard CapsNets rely on pixel-level Expectation-Maximization (EM) routing. This dense, pixel-by-pixel assignment creates a massive computational bottleneck, resulting in high parameter counts, heavy floating-point operations (FLOPs), and slow inference speeds that hinder practical deployment.

Methods

The authors propose the Mamba Capsule Routing Network (MCRNet) to transition capsule routing from the expensive pixel level to a lightweight type level.

Fig. 1: Different capsule routings for part-whole rela- tional camouflaged object detection. The original EM routing Hinton et al (2018) involves a significant num- ber of parameters and routing complexity at the pixel level due to the dense routing. Differently, our proposed MCRNet compresses spatially pixel-level capsules into type-level capsules, leading to a substantial complexity- reduction type-level capsule routing. On top of that, the capsules spatial details retrieval is used to learn the spa- tial details of mamba capsules for further camouflaged object detection.

The architecture consists of three core phases:

  1. Mamba Capsule Generation (MCG): Features extracted from a Swin Transformer backbone are converted into 2D pixel-level primary capsules. To compress these without losing global context, MCRNet utilizes Vision Mamba's multi-direction scanning mechanism (scanning in four distinct Z/N-patterns). These 1D sequences are processed by selective State Space Models (SSMs), where the final implicit latent states are extracted as abstract, type-level mamba capsules.
  2. Type-Level EM Routing: Because the spatial dimensions are compressed into type-level vectors, the subsequent EM routing algorithm operates on a fraction of the data, drastically reducing computational complexity while still successfully exploring part-whole relationships.
  3. Capsules Spatial Details Retrieval (CSDR): Operating at the type level inherently loses spatial resolution. To fix this for dense mask prediction, the CSDR module computes a cosine similarity matrix between adjacent capsule layers. This matrix acts as a guide to map high-level type relationships back onto the low-level, high-resolution primary spatial capsules.
Fig. 2: The overall architecture of MCRNet. The long-range context from Swin Transformer Liu et al (2021c) is first fed into the designed MCG module. In MCG, each type of constructed primary capsules is scanned in four directions, which are further input into the selective SSM Gu and Dao (2024) module to achieve the implicit latent state, which is treated as the type-level mamba capsules for subsequent routing to learn the high-layer mamba capsules. In the following, the proposed CSDR module is used to retrieve the spatial details of mamba capsules for final camouflaged prediction. To learn primitive object edges, the object boundary label is also taken into account for training.

Datasets

The model was trained and evaluated on three widely accepted COD benchmark datasets:

  • CAMO: 1,250 camouflaged images (1,000 training, 250 testing).
  • COD10K: A large-scale dataset consisting of 3,040 training images and 2,026 testing images.
  • NC4K: A large-scale testing dataset containing 4,121 images used exclusively to evaluate generalization capability.

Key Insights and Ideas

  • Mamba as a Capsule Abstractor: The most compelling idea is using Mamba's selective SSM hidden states to compress spatial token distributions into abstract "types." This bridges the gap between dense spatial features and abstract capsule properties.
  • Decoupled Routing and Reconstruction: Instead of forcing the routing algorithm to handle both spatial alignment and type matching simultaneously, MCRNet decouples them. Type matching happens efficiently in a compressed latent space, while spatial details are retrieved afterward via cross-layer correlation mapping.
  • Multi-Directional Context Matters: Ablation studies showed that scanning features in four directions significantly outperforms single-direction scanning, proving that spatial context preservation is vital when flattening 2D capsules into 1D Mamba sequences.

Key Results

MCRNet was compared against 25 state-of-the-art methods (including CNN, Transformer, and alternative Capsule-based networks) and achieved dominant performance across all metrics (MAE, $F_m$, $E_m$, $S_m$).

Quantitative Improvements:

  • Efficiency Gains: Compared to standard EM Routing, MCRNet reduced FLOPs from 155.16G to 145.74G, cut parameters from 77.96M to 69.11M, and decreased inference time per image from 0.039s to 0.028s.
  • Accuracy Gains: On the challenging COD10K dataset, MCRNet achieved a Mean Absolute Error (MAE) of 0.026 and a structure-measure ($S_m$) of 0.854, noticeably outperforming the previous best-performing model, VSCode.

Qualitative Findings:

Visual comparisons demonstrate that MCRNet excels at discovering small camouflaged targets, resolving highly uncertain boundaries, and maintaining object integrity even when the target is partially obscured by environmental clutter.

Limitations and Future Work

Despite its success, the authors note that MCRNet can still fail in extremely complex environments. Specifically:

  • It can be distracted by highly salient non-target objects in the background, shifting the segmentation focus away from the camouflaged object.
  • It struggles to detect very small objects located far from the center of the image due to increased observation angle distortion.

To address these issues, the authors suggest integrating the reasoning power of Large Language Models (LLMs) in future iterations to provide high-level semantic understanding of the scene context.

Summary updated

Abstract

The part-whole relational property endowed by Capsule Networks (CapsNets) has been known successful for camouflaged object detection due to its segmentation integrity. However, the previous Expectation Maximization (EM) capsule routing algorithm with heavy computation and large parameters obstructs this trend. The primary attribution behind lies in the pixel-level capsule routing. Alternatively, in this paper, we propose a novel mamba capsule routing at the type level. Specifically, we first extract the implicit latent state in mamba as capsule vectors, which abstract type-level capsules from pixel-level versions. These type-level mamba capsules are fed into the EM routing algorithm to get the high-layer mamba capsules, which greatly reduce the computation and parameters caused by the pixel-level capsule routing for part-whole relationships exploration. On top of that, to retrieve the pixel-level capsule features for further camouflaged prediction, we achieve this on the basis of the low-layer pixel-level capsules with the guidance of the correlations from adjacent-layer type-level mamba capsules. Extensive experiments on three widely used COD benchmark datasets demonstrate that our method significantly outperforms state-of-the-arts. Code has been available on https://github.com/Liangbo-Cheng/mamba\_capsule.
BibTeX
@article{zhang2025mamba,
  author = {Dingwen Zhang and Liangbo Cheng and Yi Liu and Xinggang Wang and Junwei Han},
  title = {{Mamba Capsule Routing Towards Part-Whole Relational Camouflaged Object Detection}},
  journal = {Int. J. Comput. Vis.},
  year = {2025},
  doi = {10.1007/s11263-025-02530-3},
}
Figure