Contrastive Learning for Low-Light Raw Denoising (Student Abstract)

Taoyong Cui, Yuhan DongAAAI 2024 · DOI

TL;DR

This paper introduces a novel contrastive learning approach for denoising extremely low-light raw images. By leveraging a new denoising contrastive regularization (DCR) and a custom feature embedding network (Wnet), the method effectively pulls denoised images closer to clean counterparts while pushing them away from noisy ones in a learned feature space, achieving state-of-the-art results.

What Problem Does This Paper Address?

Capturing high-quality images and videos in extremely low-light conditions is a significant challenge. While increasing exposure time or ISO can help gather more light, it also amplifies noise, leading to poor image quality. Traditional denoising methods often rely on simplistic noise models that don't accurately reflect the complex noise characteristics (quantization, bias, clipping) present in extreme low-light scenarios. Existing deep learning approaches, while better, can struggle with robust alignment in noisy scenes and often fail to effectively address color bias. This paper aims to bridge this gap by developing a more robust and effective low-light denoising method that doesn't rely on strict noise assumptions.

Methods

The core of this work is a novel Denoising Contrastive Regularization (DCR) technique. Inspired by contrastive learning in high-level vision tasks, DCR is adapted for low-level denoising. The key idea is to treat denoised images as "anchors," clean images as "positive samples," and noisy images as "negative samples." The DCR aims to pull the anchor's feature representation closer to the positive sample's and push it away from the negative sample's in a learned feature space.

A crucial component is the Wnet, a custom feature embedding network designed to extract high-frequency texture information. Unlike previous methods that might use pre-trained networks like VGG, Wnet is specifically trained for this denoising task. It utilizes Haar wavelet transforms to capture informative high-frequency components (HL, LH, HH) and then employs CNNs to learn these features. Wnet is pre-trained using a binary classification task on synthesized noisy samples to focus on noise characteristics. The DCR then leverages multi-intermediate features from Wnet, incorporating a novel loss function (Eq. 1) that combines similarities between feature maps using pixel-wise cosine similarity and L1 loss.

Datasets

The authors conducted experiments on a real low-light image/video dataset. For training data, they used a noise generator (referencing Monakhova et al. 2022) to create synthetic clean/noisy pairs. Actual noisy data was used for testing to verify the effectiveness of their method. The paper mentions using datasets from Monakhova et al. (2022) and specifically highlights experiments on "unlabeled submillilux videos."

Key Insights and Ideas

  • Contrastive Learning for Low-Level Tasks: The paper demonstrates that contrastive learning, traditionally used for high-level tasks like classification and retrieval, can be effectively adapted to improve low-level tasks like denoising, especially in challenging low-light scenarios.
  • Task-Specific Feature Extraction: Building a dedicated feature embedding network (Wnet) tailored for denoising, rather than relying on generic pre-trained models, leads to better extraction of relevant noise and texture information.
  • Exploiting Noise Structure Without Explicit Models: By using contrastive learning to learn relationships between noisy, denoised, and clean images in a feature space, the method implicitly learns to handle complex noise characteristics without needing to explicitly model them.
  • Multi-Level Feature Integration: Utilizing features from multiple layers of the Wnet enhances the contrastive learning process, allowing for a more comprehensive understanding of image features.

Key Results

The proposed method, "Baseline+DCR (Ours)," consistently outperforms existing methods across various metrics.

  • Still Images: Achieved a PSNR of 32.23, SSIM of 0.853, and LPIPS of 0.0630, outperforming baselines like "Baseline+(VGG+L1)" (30.23 PSNR, 0.842 SSIM, 0.0635 LPIPS).
  • Videos: On still images from the test set by training Modified FastDVDnet, the "Baseline+DCR (Ours)" achieved a PSNR of 30.5, SSIM of 0.889, and LPIPS of 0.060, surpassing other baselines and even a state-of-the-art method (27.7 PSNR, 0.931 SSIM, 0.078 LPIPS).
  • Qualitative Results: Figure 1 shows a visual comparison where the proposed method produces significantly cleaner and more visually appealing results on unlabeled submillilux videos compared to a baseline.

Limitations and Future Work

  • Limitations: The paper mentions that while they use a noise generator for training data, the actual noisy data is used for testing. The effectiveness of the noise generator in perfectly mimicking real-world extreme low-light noise might be a point of consideration. The reliance on "positive" and "negative" pairs, while effective, might still be sensitive to the quality of these pairs.
  • Future Work: The authors suggest that their approach can help existing benchmarks achieve better performance, implying potential for further integration and improvement on other denoising tasks. The success on low-light video denoising also opens avenues for extending this to more complex video processing tasks.

Summary updated

BibTeX
@inproceedings{cui2024contrastive,
  author = {Taoyong Cui and Yuhan Dong},
  title = {{Contrastive Learning for Low-Light Raw Denoising (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30431},
}
Figure