A Signal-dependent Video Noise Estimator Via Inter-frame Signal Suppression

Yanhao Li, Marina Gardella, Quentin Bammey, Tina Nikoukhah, Rafael Grompone von Gioi, Miguel Colom, Jean-Michel MorelImage Process. Line 2023 · DOI

Abstract

Noise estimation is a fundamental prerequisite for effective image and video processing, including denoising, forgery detection, and sensor characterization. Traditional single-frame methods often struggle with signal-dependent noise in highly textured scenes where homogeneous regions are scarce. This paper introduces a video-based noise estimation method that leverages temporal redundancy. By matching blocks across consecutive frames and analyzing their differences, the proposed algorithm effectively suppresses underlying scene content to isolate noise. We evaluate the method using a dataset of synthetic drone videos, demonstrating high accuracy (MRE 1.2%–4.5%) and robustness compared to state-of-the-art single-frame estimators.

Introduction

In the realm of digital imaging, noise is rarely uniform. Most modern sensors exhibit signal-dependent noise, where the variance of the noise changes with the intensity of the signal—a phenomenon often modeled as a combination of Poisson (shot) noise and Gaussian (read) noise. Accurately characterizing this Noise Level Function (NLF) is essential for downstream tasks.

While single-image methods rely on finding "flat" patches to measure noise, real-world video content is frequently rich in detail, leaving few such regions available. This paper addresses this limitation by moving from the spatial domain to the temporal domain, using the consistency of signals across video frames to isolate noise more effectively.

The Proposed Pipeline

The core of the proposed method is the extension of the Ponomarenko method to the temporal domain. The process involves several sophisticated steps to ensure that the estimated variance reflects actual noise rather than residual signal or matching artifacts.

Workflow of the proposed signal-dependent video noise estimation pipeline.
  1. Inter-frame Signal Suppression: The algorithm performs block matching between two consecutive frames ($FRAME_t$ and $FRAME_{t+1}$). By subtracting matched blocks, the static or slowly changing signal is cancelled out, leaving a difference block dominated by noise.
  2. The "Ring" Strategy: To prevent the algorithm from matching the noise itself (overfitting), matching is performed using a surrounding "ring" of pixels. The central block is then used for the actual noise estimation, ensuring the noise samples are independent of the matching process.
  3. Frequency Analysis and Sorting: A Discrete Cosine Transform (DCT) is applied to the difference blocks. Blocks are sorted by their low-frequency energy to identify the best-matched pairs. High-frequency coefficients are then used to estimate the noise variance.
  4. Signal-Dependent Modeling: Blocks are partitioned into intensity-based bins. This allows the algorithm to map noise variance as a function of brightness, fitting a Poisson-Gaussian model to the data.
  5. Temporal Fusion: To ensure stability across a sequence, noise curves from multiple frame pairs are combined using median fusion.

Experimental Setup

The method was rigorously tested using a dataset of 16 synthetic drone videos. These videos, originally captured in 4K, were downsampled to create "clean" references. Synthetic Poisson-Gaussian noise was added at three levels (weak, medium, and strong) to provide a ground truth for Mean Relative Error (MRE) calculations. The diverse content—ranging from urban cityscapes to natural forests—provided a challenging environment for block matching and signal suppression.

Key Insights

  • Temporal Advantage: Temporal redundancy proves to be a more powerful tool than spatial homogeneity. Even in textured scenes, the difference between frames is typically "flatter" than any individual spatial patch.
  • Block Size Considerations: Unlike spatial methods that require small blocks to find flat spots, this temporal method benefits from larger blocks (up to 47x47), which provide more high-frequency coefficients for statistical stability.
  • Subpixel Matching Trade-offs: While subpixel accuracy can improve signal suppression, it often introduces interpolation artifacts (ringing) that the estimator might mistake for noise. Consequently, subpixel matching is only recommended for very high-noise scenarios.

Results and Performance

The proposed method demonstrates superior accuracy, achieving an average MRE between 1.2% and 4.5%. In comparative tests against the standard Ponomarenko single-image method, the video-based approach showed significant gains, particularly in textured sequences where single-image methods often failed (exceeding 9% MRE).

Computationally, the algorithm is efficient enough for practical use, processing a pair of 540p frames in approximately 7.3 seconds on a standard 8-core CPU.

Limitations and Future Work

Despite its robustness, the method has limitations. It assumes noise is independent between frames, making it less effective against fixed-pattern noise or artifacts from heavy video compression (e.g., H.264). Additionally, extreme motion or rapid scene cuts can lead to overestimation if the block matching fails to align the signal perfectly. Future research may explore advanced interpolation techniques, such as Lanczos or splines, to mitigate artifacts during subpixel matching.

Summary updated

BibTeX
@article{li2023signal,
  author = {Yanhao Li and Marina Gardella and Quentin Bammey and Tina Nikoukhah and Rafael Grompone von Gioi and Miguel Colom and Jean-Michel Morel},
  title = {{A Signal-dependent Video Noise Estimator Via Inter-frame Signal Suppression}},
  journal = {Image Process. Line},
  year = {2023},
  doi = {10.5201/ipol.2023.420},
}
Figure