This paper, presented at AAAI-26, introduces 2D-CrossScan Mamba, a significant architectural refinement for State Space Models (SSMs) applied to computer vision. By moving away from the "flattened" 1D scanning patterns used in previous Mamba-based vision models, the authors propose a truly 2D-aware propagation mechanism that respects the spatial geometry of images.
TL;DR
2D-CrossScan Mamba replaces the traditional 1D scanning of images with a spatially consistent 2D framework that propagates information through multiple shortest paths simultaneously. By introducing a "cross-directional subtraction" to eliminate redundant information, it achieves superior feature representation and sets new state-of-the-art performance for SSMs on high-resolution object detection and segmentation tasks.
What Problem Does This Paper Address?
State Space Models (SSMs), particularly Mamba, have emerged as efficient alternatives to Transformers due to their linear scaling with sequence length. However, Mamba was originally designed for 1D sequences (like text). To use it for vision, researchers typically flatten 2D images into 1D sequences of patches.
This flattening creates a "spatial-locality gap." In a 1D scan, two patches that are vertically adjacent in an image might end up very far apart in the sequence. Previous attempts to fix this, like VMamba, use multi-directional 1D scans (horizontal, vertical, flipped), but these still rely on "zig-zag" paths that don't represent the shortest distance between points in 2D space. 2DMamba attempted 2D scanning but restricted communication to a single path at a time, leading to temporal gaps and inconsistent feature updates.
Methods
The authors propose three key innovations to bridge the 2D gap:
- 2D-CrossScan Framework: Instead of a 1D state equation, they reformulate the hidden state update so that a patch at coordinates $(x, y)$ aggregates information from its neighbors at $(x-1, y)$ and $(x, y-1)$ simultaneously. This ensures that information flows along the true 2D grid.
- Multi-Path Propagation & Redundancy Suppression: In a 2D grid, there are many "shortest paths" between two points. While aggregating these paths is beneficial, it leads to "redundant information accumulation" where overlapping paths over-count certain features. To solve this, the authors introduce cross-directional subtraction: they heuristically subtract the contribution of the diagonal neighbor $(x-1, y-1)$ during the update to ensure only unique information is integrated.
- Four-Corner Scanning Strategy: To ensure global context, the model performs this 2D scan starting from all four corners of the image (Top-Left, Top-Right, Bottom-Left, Bottom-Right) simultaneously. The results are then fused using four trainable weight coefficients, allowing the model to adaptively prioritize different directional flows.
Datasets
The model was evaluated on several standard and challenging benchmarks:
- PANDA: A gigapixel-level, high-resolution dataset for object detection.
- COCO: The standard benchmark for object detection and instance segmentation.
- ImageNet-1K: For image classification.
- ADE20K: For semantic segmentation.
Key Insights and Ideas
- Shortest Path Efficiency: By aligning the scanning order with the 2D spatial distance, the model ensures that "neighboring" patches are updated consecutively, preserving the geometric integrity of the image.
- The Inclusion-Exclusion Principle for Features: The "cross-directional subtraction" is a clever engineering trick. It functions similarly to the principle of inclusion-exclusion in combinatorics, preventing the feature maps from "exploding" or becoming biased toward diagonal regions where paths most frequently overlap.
- Visualizing the "Why": The Effective Receptive Field (ERF) visualizations are the "aha" moment. While VMamba produces cross-shaped artifacts (showing its 1D-scan heritage), 2D-CrossScan produces a smooth, uniform receptive field that better matches the natural distribution of information in an image.
Key Results
2D-CrossScan consistently outperformed existing SSM-based vision models with minimal increases in computational overhead:
- PANDA (Object Detection): Achieved 82.80 AP, significantly outperforming VMamba-T (80.86) and 2DMamba-T (80.93).
- COCO: Reached 47.7 APb for detection, surpassing other Mamba variants like Spatial-Mamba and LocalMamba.
- ImageNet-1K: Achieved 83.5% Top-1 accuracy, matching or exceeding more complex hierarchical models.
- Efficiency: Despite the more complex 2D logic, the model maintains a throughput comparable to 2DMamba, proving that 2D consistency doesn't have to come at the cost of speed.
Limitations and Future Work
The authors acknowledge that their redundancy suppression (the subtraction step) is currently based on a heuristic. While it works well empirically, they suggest that a more formal "unbiasedness analysis" of the state equations could lead to even more stable and theoretically grounded suppression schemes.
Additionally, while the method is highly effective for 2D images, extending this "spatially consistent" logic to 3D data (like medical scans) or video remains an open and promising direction for future research.
Summary updated