Abstract
Infrared Small Target Detection (IRSTD) is a critical task in remote sensing and surveillance, characterized by targets that are faint, small, and often obscured by complex background clutter. While Convolutional Neural Networks (CNNs) excel at local feature extraction and Vision Transformers (ViTs) capture global context, both suffer from limitations in either receptive field or computational efficiency. IRMamba introduces a novel framework that leverages the linear efficiency of State Space Models (Mamba) while addressing their inherent "local blindness." By integrating a Pixel Difference Mamba (PDMamba) for local contrast enhancement and a Layer Restoration Module (LRM) for iterative mask refinement, IRMamba achieves state-of-the-art performance with significantly reduced computational overhead.
Introduction
Detecting small targets in infrared (IR) imagery is notoriously difficult. Targets often occupy only a few pixels and lack distinct structural features, making them easily confused with sensor noise or high-frequency background elements like cloud edges.
Existing deep learning approaches face a significant trade-off. CNNs are efficient but often fail to distinguish small targets from large-scale background structures due to limited receptive fields. Conversely, ViTs provide the necessary global context but exhibit quadratic computational complexity, which is prohibitive for high-resolution or real-time applications. Mamba architectures offer a promising middle ground with linear complexity and global reach; however, they typically process images as 1D sequences, which destroys the 2D spatial dependencies—specifically the local contrast—essential for identifying tiny infrared targets.
Architecture Overview
IRMamba is designed as an encoder-decoder network that bridges the gap between global context and local precision. The encoder utilizes a series of Down-Sample layers and PDMamba blocks to extract multi-scale features, while the decoder employs Patch Expansion and ResNet blocks, capped by an iterative Layer Restoration Module (LRM).
Methodology
Pixel Difference Mamba (PDMamba)
To overcome the spatial limitations of standard State Space Models (SSMs), the authors developed the Pixel Difference Mamba (PDMamba). Standard SSMs flatten 2D images into 1D, losing the directional contrast information vital for IRSTD. PDMamba modifies the core SSM equation by incorporating central pixel differentiation. By calculating the horizontal and vertical differences between a pixel and its neighbors, the model injects local spatial awareness directly into the global state equation. This allows the network to maintain a global "memory" of the scene while remaining hyper-sensitive to the local intensity variations that characterize a small target.
Layer Restoration Module (LRM)
Rather than treating detection as a single-step segmentation task, IRMamba frames the final stage as an iterative image restoration problem. Inspired by "Double Deep Image Prior" theory, the LRM treats the input as a combination of a target layer and a background/noise layer.
As shown in the figure above, the module utilizes Bidirectional Information Transmission Gradient Descent (BITGD) to iteratively refine the target mask. By gradually "peeling away" the background noise across multiple stages (typically four), the LRM produces a high-confidence, low-noise detection mask that significantly reduces false alarms.
Experimental Results
IRMamba was evaluated on three major IRSTD benchmarks: NUAA-SIRST, NUDT-SIRST, and IRSTD-1K.
- State-of-the-Art Accuracy: On the NUDT-SIRST dataset, IRMamba achieved an Intersection over Union (IoU) of 95.18, outperforming previous leaders like SCTransNet (94.09).
- Low False-Alarm Rates: The model demonstrated superior background suppression, achieving a False-Alarm Rate ($F_a$) of 0.887 on NUAA-SIRST, significantly lower than many CNN-based counterparts.
- Efficiency: With only 10.51M parameters, IRMamba is roughly five times smaller than UIU-Net (50.54M), while maintaining faster inference speeds and lower memory consumption as image resolution scales.
Conclusion
IRMamba represents the first successful adaptation of the Mamba architecture for the IRSTD task. By introducing pixel-level differentiation into the SSM framework and adopting a restoration-based approach to mask generation, the model effectively balances global context with local sensitivity. This approach not only sets a new performance benchmark but also provides a computationally efficient path forward for high-precision infrared surveillance and remote sensing.
Summary updated