Infrared small-target detection algorithms: a survey

Junhong Li, Ping Zhang, Xiaowei Wang, Shize HuangJournal of Image and Graphics 2020 · DOI

TL;DR

Detecting tiny, dim targets in messy infrared (IR) images is a high-stakes game of hide-and-seek essential for military and industrial safety. This survey provides a comprehensive roadmap of the field, categorizing methods into single-frame and sequence-based approaches while highlighting the shift from simple spatial filters to sophisticated "low-rank" mathematical modeling and emerging deep learning techniques.

What Problem Does This Paper Address?

Infrared small-target detection is the "needle in a haystack" problem of computer vision. Unlike standard photography, IR targets often appear as mere blobs occupying fewer than $10 \times 10$ pixels, lacking any distinct shape, color, or texture.

The authors identify three primary challenges:

  1. Low Signal-to-Clutter Ratio (SCR): Targets are often buried in "clutter" like clouds, sea waves, or sensor noise, making them nearly invisible.
  2. High Mobility: Targets move fast and unpredictably, requiring high-precision tracking.
  3. Complex Backgrounds: IR sensors are sensitive to environmental heat, meaning a sunny cloud can look more like a "target" than the actual object of interest.

Traditional visible-light detection algorithms fail here because they rely on features (like edges or parts) that simply don't exist in a 5-pixel IR blob.

Methods

The paper categorizes detection algorithms into two main families:

1. Single-Frame Detection

These methods look at one image at a time.

  • Filtering-based: The "old guard." These use spatial filters (like Top-Hat) or frequency domain transforms (like Wavelets) to suppress the background. They are fast but struggle with non-uniform, complex backgrounds.
  • Human Visual System (HVS): These mimic how the human eye spots anomalies. Algorithms like Local Contrast Method (LCM) calculate how much a pixel stands out from its immediate neighbors.
  • Low-Rank and Sparse Recovery: The current state-of-the-art. These treat the background as a "low-rank" matrix (highly redundant) and the target as a "sparse" component (an outlier). Methods like IPI (Infrared Patch-Image) and NRAM use heavy-duty optimization to separate the two.

2. Sequence-Based Detection

These leverage temporal information across multiple frames.

  • Detect Before Motion (DBM): Detect candidates in each frame first, then use motion consistency to filter out false alarms.
  • Motion Before Detect (MBD): Accumulate energy over time (e.g., via 3D Matched Filtering or Particle Filters) before making a detection. This is much better for extremely "weak" targets that are invisible in a single frame.

Datasets

The paper highlights a major bottleneck: the lack of public datasets. Due to the military sensitivity of IR data, there is no "ImageNet" for this field.

  • Researchers often use private sequences or simulate targets by overlaying small Gaussian blobs onto real IR backgrounds.
  • The authors evaluated eight representative algorithms across five different sequences (sky, sea, and land backgrounds) to provide a fair objective comparison.

Key Insights and Ideas

  • The Sparsity Revolution: The move toward "Low-Rank + Sparse" decomposition has been the biggest leap in performance. By mathematically defining the background as "predictable" and the target as "unpredictable," researchers can find targets even in very noisy scenes.
  • HVS Efficiency: HVS-based methods offer a "sweet spot." They are significantly faster than complex optimization models while being much more robust than simple filters.
  • The Real-Time Paradox: The best-performing algorithms (like NRAM) are often the slowest. For a missile or a drone, a high-accuracy algorithm that takes 2 seconds to process a frame is useless; it needs to happen in milliseconds.

Key Results

The authors compared 8 algorithms using metrics like Signal-to-Clutter Ratio Gain (SCRG) and Background Suppression Factor (BSF):

  • Top-Hat (Filtering): Fastest ($O(k^2 \log k^2 MN)$) but lowest performance in complex scenes.
  • NRAM/SMSL (Low-Rank): Highest detection accuracy and best background suppression, but extremely high computational complexity ($O(MN^2)$).
  • LCM/MPCM (HVS): Balanced performance, suitable for scenarios where hardware resources are moderate.

Limitations and Future Work

The survey concludes with several "open frontiers":

  1. Deep Learning Gap: While DL is dominant in visible light, it struggles in IR due to the lack of training data. Future work involves using GANs (Generative Adversarial Networks) to synthesize realistic training data.
  2. Hardware Implementation: There is a desperate need for algorithms that can be easily ported to FPGAs or DSPs for real-time use.
  3. Multi-Sensor Fusion: Combining IR with visible light or radar data to overcome the limitations of any single sensor.
  4. Robustness: Most current algorithms are tuned for specific backgrounds (e.g., just sky). Developing "universal" detectors that work across sea, land, and air remains a challenge.

Summary updated

Abstract

红外探测技术具有不受环境等因素干扰的优势,在红外制导、预警等军事领域的应用日益广泛。随着对红外弱小目标检测技术的研究越来越深入,相应的检测方法越来越多样。本文通过对红外弱小目标图像中目标与背景的特性以及红外弱小目标检测技术难点问题进行分析,根据当前是否利用帧间相关信息,分别从基于单帧红外图像和基于红外序列两个角度,选取了相应的红外弱小目标算法进行对比,对其中典型算法的原理、流程以及特点等进行了详细综述,并对每类检测算法的性能进行了比较。针对红外弱小目标图像信噪比低的特点,对红外弱小目标检测算法的难点问题进行分析,给出了目前各种算法的解决方法和不足,探讨红外弱小目标检测算法的发展方向,即研究计算量小、性能优、鲁棒性强、实时性好和便于硬件实现的算法。
BibTeX
@article{li2020infrared,
  author = {Junhong Li and Ping Zhang and Xiaowei Wang and Shize Huang},
  title = {{Infrared small-target detection algorithms: a survey}},
  journal = {Journal of Image and Graphics},
  year = {2020},
  doi = {10.11834/jig.190574},
}
Figure