State Space Models (SSMs) like Mamba and RWKV have emerged as efficient alternatives to Transformers, offering linear computational complexity. However, these models are inherently one-dimensional and causal—they process data as a sequence where each element only "sees" what came before it. When applied to two-dimensional images or three-dimensional medical volumes, this 1D nature creates a fundamental mismatch: flattening an image into a string of pixels destroys spatial adjacency and introduces a "directional bias" that ignores the non-causal nature of visual data.
To bridge this gap, researchers have developed multi-directional scanning strategies. These techniques transform 1D sequence models into spatially aware architectures by traversing the data through multiple paths, ensuring that every pixel can integrate context from its entire neighborhood.
The Spatial Discontinuity Problem
In a standard 1D scan (like row-major order), a pixel is adjacent to its left and right neighbors in the sequence. However, its vertical neighbor—the pixel directly above it—might be hundreds of tokens away. For a recurrent model or an SSM, this distance makes it difficult to capture vertical textures or maintain global consistency.
Yan Yan and Bin Xie identify this as the "spatial discontinuity" and "high-variance" issue in MM-UNet. When 3D medical data is linearized, the boundaries of the sequence lose critical context. By processing the flattened data in both forward and reverse directions, the model provides the missing context at the sequence boundaries, significantly improving segmentation accuracy.
Reconstructing the 2D Receptive Field
The most common solution to the dimensionality mismatch is the Cross-Scan or Four-Directional Scan. By scanning an image horizontally (left-to-right and right-to-left) and vertically (top-to-bottom and bottom-to-top), the model ensures that every pixel acts as a "hub" where information from all four cardinal directions converges.
Di Wang and Lefei Zhang utilize this in MambaMoE to preserve spatial relationships in hyperspectral images. This approach mitigates the loss of fine-grained structure inherent in single-pass processing. Similarly, Juan Wen and Luc Van Gool show in MatIR that multi-directional paths allow linear-time models to achieve spatial coverage comparable to 2D convolutions, effectively expanding the effective receptive field (ERF).
Beyond Cardinal Directions
While four directions cover the axes, they can still leave "blind spots" along the diagonals. Recent work has pushed toward Omni-directional Scanning to capture more complex morphologies.
- Eight-Directional Scanning: Haodong Pan and Nanning Zheng introduced an eight-direction pattern in HCFSSNet that includes diagonal paths. This ensures that every pixel interacts with its immediate first-order neighbors, preventing the loss of diagonal structural information common in 1D flattening.
- Diagonal Skewing: In the early PixelRNN, Aäron van den Oord proposed a diagonal scanning pattern by skewing the input feature maps, ensuring the prediction for any pixel incorporates information from all preceding pixels in the spatial topology.
- Spiral and S-Shaped Trajectories: For specific domains like medical imaging, Omer F. Atli and Bilal Kabas developed an isotropic spiral-scan in I2I-Mamba. By moving from the center outward, the model balances its receptive field across all directions, which is particularly effective for non-axial anatomical structures. Boyun Li further refined this in MaIR using "Nested S-shaped Scanning" to preserve local continuity.
Efficiency through Inter-Layer Alternation
Running four or eight scans in every layer is computationally expensive. To maintain the speed advantage of SSMs, some researchers propose distributing the scanning directions across the depth of the network rather than within a single block.
In Adventurer, Feng Wang and Yaodong Yu use inter-layer sequence flipping. Instead of multi-directional scans in one layer, they reverse the sequence order between consecutive layers. This allows patches at the end of the sequence in Layer $N$ to be at the beginning in Layer $N+1$, facilitating bidirectional context over the depth of the network. Benedikt Alkin employs a similar logic in Vision-LSTM, alternating the processing direction between successive layers to integrate global information using causal operators.
Why It Works: Receptive Field and Inductive Bias
The intuition behind multi-directional scanning is two-fold:
- Eliminating Directional Bias: Causal models naturally "forget" or down-weight early tokens. By scanning from multiple directions, the "forgotten" regions in one pass become the "prioritized" regions in another. Yu‐Cheng Lin demonstrates in EAMamba that diagonal paths specifically address "local pixel forgetting" and eliminate spatial blind spots.
- Manhattan Distance vs. Sequence Distance: Standard 1D SSMs calculate dependencies based on distance in the flattened string. Jingwei Zhang and Dimitris Samaras argue in 2DMamba that 2D scanning allows the model to calculate dependencies based on Manhattan distance (spatial steps) rather than sequence distance. This results in a smoother, more globally consistent effective receptive field, removing the "cross-shaped" artifacts often seen in naive 1D-to-2D adaptations.
By treating the image not as a single string, but as a multi-directional intersection of sequences, these architectures allow linear-complexity models to achieve the spatial awareness of Transformers and the local inductive bias of CNNs.