Temporal-Spatial Nonlinear Filtering for Infrared Focal Plane Array Stripe Nonuniformity Correction

Jia Li, Hanlin Qin, Xiang Yan, Qingjie Zeng, Tingwu YangSymmetry 2019 · DOI

TL;DR

This paper introduces a novel temporal-spatial nonlinear filtering method to effectively correct stripe nonuniformity in infrared images. By combining a weighted guided image filter for spatial estimation with a nonlinear diffusion equation for temporal refinement, the method achieves superior correction performance and detail preservation compared to existing techniques, requiring only a few frames for processing.

What Problem Does This Paper Address?

Infrared focal plane array (IRFPA) detectors are prone to stripe nonuniformity, an artifact that degrades image quality and hinders the performance of critical applications like surveillance and early warning systems. Traditional calibration-based methods are susceptible to detector drift and require frequent recalibration. While scene-based methods offer adaptivity, existing approaches often suffer from ghosting artifacts, slow convergence, or insufficient detail preservation. This paper aims to address these limitations by developing an adaptive and efficient method that leverages both spatial and temporal information to accurately remove stripe nonuniformity while preserving image details.

Methods

The proposed method comprises two main stages:

  1. Spatial-Domain Nonuniformity Coarse Estimation: This stage utilizes a Weighted Guided Image Filter (WGIF) with a newly designed global weight map. The weight map is crucial for distinguishing between stripe noise and important image features like edges and targets. WGIF filters the input image to produce an "ideal" image, and the difference between the original and ideal image yields a coarse estimation of the stripe nonuniformity. The novel weight map is designed to assign higher weights to edges and stable backgrounds, thus protecting them from smoothing, while assigning lower weights to stripe noise for effective removal.
  2. Temporal-Domain Nonuniformity Fine Estimation: This stage refines the nonuniformity estimation by exploiting the temporal characteristics of stripe noise and scene residuals. A nonlinear diffusion equation is employed to process the temporal profiles of pixels across consecutive frames. The key insight here is that stripe nonuniformity changes slowly over time, while scene residuals (like moving objects) exhibit more drastic fluctuations. The diffusion equation effectively separates these two components, leading to a more accurate estimation of the true nonuniformity.

Finally, the accurately estimated nonuniformity is subtracted from the original infrared video frames to obtain the corrected output.

Datasets

The paper evaluates its method on four representative real infrared image sequences obtained from an uncooled infrared camera. These sequences are described as:

  • Sequence 1: Stable background with a walking girl (180x200 pixels).
  • Sequence 2: Small target moving at high velocity against a complex cloud background (256x256 pixels).
  • Sequence 3: Moving camera scene with abundant details (128x128 pixels).
  • Sequence 4: Scene with strong horizontal and vertical edges (150x200 pixels).

The paper does not introduce new datasets but uses these existing sequences for rigorous testing and comparison.

Key Insights and Ideas

  • Temporal vs. Spatial Characteristics: The core idea is that stripe nonuniformity has distinct temporal (slowly changing) and spatial (vertical stripes) characteristics that can be exploited. Scene residuals, on the other hand, can have different temporal behaviors (e.g., rapid changes due to motion).
  • Adaptive Weighting for Detail Preservation: The novel global weight map in the WGIF is key. It allows the filter to be more aggressive in smoothing noise while being conservative around edges and important details, a common challenge in image filtering.
  • Nonlinear Diffusion for Temporal Separation: Using a nonlinear diffusion equation to process temporal profiles is an elegant way to separate slow-changing nonuniformity from faster-changing scene elements, leading to a cleaner nonuniformity estimate.
  • Efficiency with Few Frames: A significant advantage is that the method achieves good results using only a small number of consecutive frames, making it practical for real-time applications.

Key Results

  • Qualitative Improvement: The proposed method visually outperforms traditional and deep learning-based NUC methods in suppressing stripe nonuniformity while preserving image details, including small targets and edges. Figures demonstrate a noticeable reduction in artifacts and a clearer image compared to other methods.
  • Quantitative Metrics: The method achieves smaller values for residual nonuniformity (U) and roughness (ρ) compared to other tested methods across various sequences. For instance, in Sequence 1, U and ρ were reduced by 38.9% and 71.6% lower, respectively, compared to the original image.
  • Efficiency: The method requires only a fraction of the video frames (less than ten) for effective correction.
  • Computational Time: The proposed method, when using its own weight map, is competitive in terms of computing time, often faster than methods using weights from prior literature.

Limitations and Future Work

  • Computational Cost of WGIF: While the overall method is efficient, the WGIF stage, especially with certain weight maps, can be time-consuming. The paper notes that the proposed weight map is more time-efficient than some prior methods.
  • Parameter Tuning: Like many image processing algorithms, the performance might be sensitive to parameter settings, though the paper provides specific parameters for their experiments.
  • Future Work: The paper doesn't explicitly detail future work, but the success of combining spatial filtering with temporal diffusion suggests potential for further exploration of advanced diffusion models or more sophisticated spatial-temporal feature extraction techniques for even more robust and adaptive nonuniformity correction.

Summary updated

BibTeX
@article{li2019temporal,
  author = {Jia Li and Hanlin Qin and Xiang Yan and Qingjie Zeng and Tingwu Yang},
  title = {{Temporal-Spatial Nonlinear Filtering for Infrared Focal Plane Array Stripe Nonuniformity Correction}},
  journal = {Symmetry},
  year = {2019},
  doi = {10.3390/sym11050673},
}
Figure