A Real-Time Cascaded Video Denoising Algorithm Using Intensity and Structure Tensor

Xin Tan, Yu Liu, Huaxin Xiao, Maojun Zhang

Abstract

Video denoising in low-light environments presents a significant challenge, as sensor noise often obscures motion, leading to artifacts like ghosting or excessive blurring. This paper introduces a real-time denoising algorithm specifically designed for stationary background scenes, such as those found in surveillance. The proposed method utilizes a novel change segmentation approach that combines pixel intensity with structure tensors to robustly distinguish motion from noise. Implemented in a three-stage cascaded framework, the algorithm iteratively refines the video quality. Experimental results demonstrate that the method outperforms state-of-the-art techniques like VBM3D in high-noise scenarios while maintaining the low latency required for FPGA-based real-time processing.

Introduction

Video denoising is a critical task in computer vision, particularly for surveillance systems operating in low-light conditions. The goal is to suppress sensor noise without compromising image sharpness or introducing motion artifacts. While temporal frame averaging is highly effective for static scenes, it often results in "ghosting" when objects move. Traditional motion compensation techniques rely on intensity changes to detect movement, but these methods frequently fail when noise levels are high. Furthermore, many top-tier algorithms, such as VBM3D, are computationally prohibitive for real-time hardware implementation due to their complexity and reliance on future frames.

Methodology

The proposed solution addresses these challenges through two primary innovations: a hybrid segmentation approach and a cascaded processing framework.

Intensity and Structure Tensor Segmentation

To accurately identify motion in noisy frames, the algorithm combines traditional intensity-based change detection with structure tensors. Structure tensors analyze local geometric features, such as edges and corners, which are more resilient to noise than raw pixel values. By calculating the distance between tensors using a Log-Euclidean metric, the system can distinguish between actual physical movement and random sensor fluctuations.

Comparison of change detection using intensity (row a), blurred intensity (row b), and structure tensors (row c).

The Cascaded Framework

The algorithm employs a three-stage pipeline where each stage iteratively cleans the video. The denoised output of the first stage serves as the "pre-filtered" input for the second, and so on. This iterative approach allows the motion detection logic to operate on increasingly cleaner data, theoretically suppressing noise to $1/N^3$ of its original level by the final stage.

The proposed three-stage cascaded denoising architecture.

Experimental Results

The algorithm was tested using standard grayscale sequences, including Salesman and Hall Monitor, under varying levels of additive Gaussian white noise ($\sigma_n$ from 5 to 100).

Key findings include:

  • Performance at High Noise: The method excels in extreme conditions, outperforming VBM3D and SURE-LET for $\sigma_n \ge 50$. In the Salesman test at $\sigma_n = 100$, the proposed algorithm achieved a PSNR of 24.44 dB, compared to 23.16 dB for VBM3D.
  • Visual Quality: Qualitative analysis shows superior preservation of fine textures and edges, avoiding the "mushy" over-smoothed appearance common in other high-noise filters.
  • Real-Time Efficiency: Designed as a causal system (using only current and past frames), the algorithm was successfully implemented on a Xilinx Virtex-5 FPGA. It processes 1080p video at 30 fps with a total latency of only 3.33 ms.

Conclusion

By integrating geometric structure analysis with a cascaded refinement architecture, this algorithm provides a robust and hardware-efficient solution for video denoising. While currently optimized for stationary backgrounds, its ability to handle extreme noise levels in real-time makes it highly suitable for practical security and industrial applications. Future work may involve extending the framework to handle non-stationary backgrounds and more complex noise models.

Summary updated

BibTeX
@article{tan2015real,
  author = {Xin Tan and Yu Liu and Huaxin Xiao and Maojun Zhang},
  title = {{A Real-Time Cascaded Video Denoising Algorithm Using Intensity and Structure Tensor}},
  journal = {IEICE Trans. Information and Systems},
  year = {2015},
  doi = {10.1587/transinf.2014edp7435},
}
Figure