TL;DR
This paper introduces a novel two-stage filtering method for correcting stripe nonuniformity in single infrared (IR) images. By leveraging the spectral characteristics of stripe noise, the method effectively separates image structures from noise, leading to superior correction performance and better preservation of image details compared to existing state-of-the-art techniques.
What Problem Does This Paper Address?
Infrared (IR) imaging is crucial for various applications like target detection and face recognition. However, IR images often suffer from fixed pattern noise (FPN), particularly stripe nonuniformity, which degrades image quality and hinders downstream analysis. Existing methods for correcting this noise often struggle to balance noise removal with the preservation of fine image details, leading to over-smoothing or artifacts. This paper tackles the challenge of correcting stripe nonuniformity in a single IR image without relying on multiple frames, aiming to achieve high-quality correction while retaining essential image structures.
Methods
The proposed method employs a two-stage filtering strategy:
- Stage 1: Spectral Filtering with a Stripe Notch Filter (SNF):
- The IR image is transformed into the frequency domain using a 2D Fast Fourier Transform (FFT).
- The authors analyze the spectral characteristics of stripe noise, observing a distinct bright line in the Fourier spectrum that is orthogonal to the stripe direction. This forms the basis for the Stripe Frequency Band (SFB).
- A Stripe Notch Filter (SNF) is designed to reject frequencies within the identified SFB. This filter effectively separates the image structure from the stripe noise.
- An inverse FFT is then applied to reconstruct a "structure layer" (I₁) that contains the image details but lacks grayscale information.
- Stage 2: Spatial Filtering with a 1-D Row Smoothing Filter:
- The residual image (R = Original Image - Structure Layer) contains the stripe noise and some grayscale information.
- A 1-D row smoothing filter, implemented iteratively, is applied to this residual image. This stage aims to estimate and refine the "grayscale layer" (I₂) by removing the stripe noise while preserving the grayscale information.
- The final corrected image is obtained by adding the structure layer (I₁) and the grayscale layer (I₂).
The method combines spectral analysis for initial noise separation with spatial filtering for fine-tuning, aiming for a coarse-to-fine correction process.
Datasets
The paper evaluates its method on two types of datasets:
- Simulated Images: Standard grayscale images (Lena and Mandrill) were artificially corrupted with simulated stripe noise at various levels (controlled by parameter σ).
- Raw IR Images: Three real-world IR images were used:
- "Street" from the ASL dataset.
- "Building" from Tendero's dataset.
- "Kettle" from the authors' own dataset.
Key Insights and Ideas
- Spectral Signature of Stripes: The core insight is that stripe noise has a distinct spectral signature in the frequency domain, appearing as a bright line orthogonal to the stripe direction. This allows for effective separation from image structures.
- Two-Stage Approach: Combining spectral filtering (for coarse separation) with spatial filtering (for fine-tuning and detail preservation) is crucial for achieving high-quality correction without artifacts.
- Stripe Frequency Band (SFB): Identifying and utilizing the SFB as a rejection region in the spectral filter is key to isolating stripe noise.
- Grayscale Information in Residual: The residual image after spectral filtering still contains important grayscale information, necessitating a spatial filtering stage to recover it accurately.
- Detail Preservation: The method's ability to preserve image details is attributed to the careful separation of structural information in the first stage and the iterative refinement in the second.
Key Results
- Quantitative Evaluation: The proposed method consistently achieved the highest Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) scores on both simulated and raw IR images compared to four state-of-the-art methods (WTSF, MIRE, GIF, CNN). For example, on Lena with σ=0.16, the proposed method achieved a PSNR of 16.61 dB and SSIM of 0.932, outperforming others.
- Qualitative Evaluation: Visual results demonstrated that the proposed method effectively removed stripe artifacts while preserving image details better than competing methods. Methods like GIF and CNN sometimes left visible artifacts or caused unnatural grayscale degradation.
- Raw IR Image Performance: The method showed strong performance on real IR images, even those with heavy nonuniformity (e.g., "Building").
- NUN Analysis: Nonuniformity Noise (NUN) images (difference between original and corrected) showed minimal structure information for the proposed method, indicating effective noise removal without introducing new artifacts.
- Computational Efficiency: The method demonstrated competitive or superior computational speed compared to other methods, suggesting its potential for real-time applications.
Limitations and Future Work
- Parameter Sensitivity: The performance is influenced by parameters like the width of the SFB rejection region (K) and the number of spatial filtering iterations (imax). While K=2 and imax=10 were found to be generally good, optimal values might vary.
- Over-Suppression Artifacts: In some cases, especially with similar patterns to stripes, the method might lead to over-suppression and minor artifacts.
- Future Work:
- Further exploration of the spectral distribution of stripe noise.
- Developing adaptive methods for detecting stripe frequencies.
- Incorporating more prior knowledge of IR images and object scenes for improved correction.
Summary updated