TL;DR
This paper introduces an improved nonuniformity correction (NUC) algorithm for cooled infrared imagers that specifically targets "sky ripple" noise. By using fuzzy scene classification to adaptively adjust correction thresholds, the method effectively removes this noise while preserving image details and avoiding ghosting artifacts, outperforming existing techniques. The algorithm has also been successfully implemented on an FPGA for real-time processing.
What Problem Does This Paper Address?
Infrared (IR) imagers, particularly cooled ones, often suffer from a specific type of noise called "ripple residual nonuniformity" (RNU) when observing sky scenes. This noise is characterized by a low-frequency, spatially continuous ripple that significantly degrades image quality, making it difficult to detect small targets. Traditional calibration-based NUC methods struggle with this noise, especially when the observed radiation falls outside the calibration range (common in sky observations). Existing scene-based NUC algorithms also face challenges, either being computationally expensive, prone to ghosting artifacts, or having difficulty adapting to the variable nature of ripple RNU. The core problem is to effectively remove this specific type of noise without sacrificing crucial image details or introducing new artifacts, especially for applications like small target detection.
Methods
The paper proposes an Improved Temporal High-Pass & Grayscale Mapping (ITHP & GM) algorithm. This method builds upon the existing THP & GM algorithm by incorporating a fuzzy scene classification module.
Here's a breakdown of the key components:
- Adaptive Thresholding: The core innovation is the adaptive control of correction thresholds. Instead of using fixed thresholds for temporal (Tte) and spatial (Tsp) filtering, these thresholds are dynamically adjusted based on the scene content.
- Fuzzy Scene Classification: A fuzzy logic system is employed to classify the scene into three categories: SKY, HALF-SKY, and GROUND. This classification is based on statistical properties of image blocks, such as mean grayscale and grayscale differences between blocks. The output of this classifier is a "sky similarity" value.
- Threshold Adjustment: The "sky similarity" value directly influences the temporal and spatial thresholds. Higher sky similarity leads to larger thresholds, enabling more aggressive correction for ripple RNU in sky-like scenes. Conversely, lower sky similarity results in smaller thresholds, preserving details in ground scenes and preventing ghosting.
- Hardware Implementation: The algorithm has been implemented on a Xilinx Virtex-5 XC5VLX50T FPGA. This implementation focuses on efficient processing with low resource consumption and minimal hardware delay (less than 10 image rows).
Datasets
The paper uses real-world infrared video sequences for evaluation.
- Scene Classification Dataset: A collection of infrared images from August 2015, including deep-sky, mid-sky, near-ground-sky, ground, and indoor scenes. Each scene type has 5000 frames.
- RNU Correction Dataset: A sequence of 450 frames from a calibrated infrared imager (320x256 detector, 16-bit, 50 fps) that captured outdoor scenes, with ripple RNU appearing around the 260th frame.
- Pure Sky Dataset: A sequence of 520 pure sky images with almost uniform radiation, some containing small flying objects, used to evaluate convergence rate.
The paper also mentions using a separate image sequence of 9044 frames collected in June 2016 for further verification of the fuzzy classifier's accuracy.
Key Insights and Ideas
- The "Sky Ripple" is a Distinct Problem: The paper clearly identifies ripple RNU as a specific challenge for cooled IR imagers observing the sky, distinct from general FPN.
- Adaptive Thresholding is Crucial: The success of the proposed method hinges on the adaptive control of correction thresholds. Fixed thresholds, as used in prior methods, lead to either insufficient noise removal or significant detail degradation.
- Fuzzy Logic for Robust Scene Understanding: Fuzzy logic provides a flexible and resilient way to classify scenes, especially for transitional "half-sky" scenarios, which is critical for accurate threshold adaptation. This is more effective than simple binary classification.
- Trade-off Management: The core idea is to manage the inherent trade-off between noise suppression and detail preservation by dynamically adjusting the correction aggressiveness based on the scene's sky-like characteristics.
- FPGA Implementation for Real-Time Needs: The successful hardware implementation demonstrates the algorithm's suitability for real-time applications in infrared imaging systems, offering low latency and efficient resource usage.
Key Results
- Improved Detail Preservation: The ITHP & GM method significantly preserves image details compared to ST-LMS and THP & GM, as evidenced by visual inspection and the red rectangular areas highlighting preserved details in Figure 12.
- Effective Ripple RNU Removal: The algorithm effectively removes ripple RNU, achieving a similar or better noise reduction capability than existing methods.
- Faster Convergence: The method shows a faster convergence rate for ripple RNU correction, as indicated by the NU curves in Figure 16.
- Reduced Ghosting Artifacts: The adaptive thresholding successfully suppresses ghosting artifacts, which are present in other methods (e.g., ST-LMS in Figure 12b).
- High Scene Classification Accuracy: The fuzzy classifier achieved high accuracy for sky (98.9%) and ground (99.9%) scenes, with slightly lower accuracy for half-sky (98.3%), which is noted as a potential area for improvement.
- FPGA Performance: The FPGA implementation offers low resource consumption (e.g., 1600 slices, 7 DSP48E) and a small hardware delay (<10 image rows), making it suitable for real-time processing.
Limitations and Future Work
- Half-Sky Classification Accuracy: The paper acknowledges that the classification accuracy for half-sky images is the lowest. This can lead to misclassification, potentially causing detail loss in ground portions of half-sky images or insufficient RNU removal if misclassified as ground.
- Parameter Tuning: The fuzzy classifier's performance relies on the correct setting of parameters like T1 and T2. The paper mentions that the thermal imager needs to be aimed at the sky at boot time for initial calculation of these parameters.
- Future Work: The authors suggest future work could focus on "how to more accurately differentiate the ripple RNU from the scene," implying further refinement of the scene classification to better distinguish between true ripple RNU and other scene features, especially in complex half-sky scenarios.
Summary updated