Image Deraining and Denoising Convolutional Neural Network ForAutonomous Driving

Kaige Wang, Long Chen, TIanming Wang, Qixiang Meng, Huatao Jiang, Lin ChangarXiv 2021 · DOI · arXiv

Abstract: Perception plays an important role in reliable decision-making for autonomous vehicles. Over the last ten years, huge advances have been made in the field of perception. However, perception in extreme weather conditions is still a difficult problem, especially in rainy weather conditions. In order to improve the detection effect of road targets in rainy environments, we analyze the physical characteristics of the rain layer and propose a deraining convolutional neural network structure. Based on this network structure, we design an ablation experiment and experiment results show that our method can effectively improve the accuracy of object detection in rainy conditions.

Introduction

Perception systems for autonomous vehicles (AVs) perform remarkably well under "ideal" sunny conditions, but their reliability plummets during heavy rain. Rain creates a multi-layered problem: nearby raindrops obstruct and distort objects, distant rain bands create a fog-like blur, and the low-light/high-moisture environment introduces significant sensor noise. Existing "model-driven" computer vision techniques are often too slow for real-time driving or fail to produce clean enough images. This paper seeks to bridge that gap by creating a network that understands the distinct physical characteristics of the rain layer versus random sensor noise.

Proposed Methodology

The authors propose a hierarchical cascaded neural network based on the physical model: $$Y = X + R + Noise$$ Where $Y$ is the rainy image, $X$ is the clean image, $R$ is the rain layer, and $Noise$ is the random environmental noise.

Key architectural features include:

  • Two-Stage Extraction: The network is split into an upper section that extracts the rain layer ($R$) and a lower section that extracts the noise layer ($Noise$).
  • Residual Learning: Both stages utilize residual blocks to focus on learning the high-frequency differences (the rain and noise) rather than the entire image content.
  • Pixel-Shuffle/Unshuffle: These layers are used to accelerate computation, allowing the network to process high-resolution images more efficiently.
  • Subtraction Logic: Instead of trying to "generate" a clean image directly, the network predicts the rain and noise components and subtracts them from the original input ($X = Y - R - Noise$).
The proposed hierarchical cascaded network architecture for sequential rain and noise removal.

Datasets and Evaluation

The model was trained and evaluated using a mix of synthetic and real-world data. Synthetic benchmarks like Rain100L and Rain100H provided pairs of "clean" and "automatically rained-on" images. Additionally, the authors collected a real-world dataset using manual shooting and fixed-point cameras to capture actual rainy scenes, ensuring the model generalizes to unpredictable weather.

Key Insights

A critical finding is that rain is more than just streaks; treating rain and noise as separate high-frequency components is essential. A network designed only to remove rain streaks often leaves behind a "grainy" image that confuses object detectors. Through ablation studies, the authors proved that the synergy between the deraining and denoising stages drives the performance gain. The ultimate goal is a "machine-readable" image that improves downstream tasks like YOLO v3 object detection.

Results

Visual comparison showing the input rainy image (left) and the output after processing through the proposed network (right). Source, p. 2

Visual comparison showing the input rainy image (left) and the output after processing through the proposed network (right).

The proposed network was compared against state-of-the-art models (DDN, JDNet, MPRNet):

  • PSNR (Peak Signal-to-Noise Ratio): Reached 37.52, a 14.36% improvement over MPRNet.
  • SSIM (Structural Similarity Index): Achieved a score of 0.9745, outperforming all competitors.
  • Object Detection: Preprocessing directly fixed common YOLO v3 errors, such as misclassifying a motorcycle as a human due to rain distortion, and increased confidence levels for detected vehicles.

Conclusion and Future Work

While highly effective for small and medium raindrops, the network shows a "weakening effect" rather than total removal for very large raindrops due to significant occlusion. Future work will focus on improving real-world training datasets and optimizing the model to handle other extreme conditions like heavy fog or snow.

Summary updated

Abstract

Perception plays an important role in reliable decision-making for autonomous vehicles. Over the last ten years, huge advances have been made in the field of perception. However, perception in extreme weather conditions is still a difficult problem, especially in rainy weather conditions. In order to improve the detection effect of road targets in rainy environments, we analyze the physical characteristics of the rain layer and propose a deraining convolutional neural network structure. Based on this network structure, we design an ablation experiment and experiment results show that our method can effectively improve the accuracy of object detection in rainy conditions.
BibTeX
@misc{wang2021image,
  author = {Kaige Wang and Long Chen and TIanming Wang and Qixiang Meng and Huatao Jiang and Lin Chang},
  title = {{Image Deraining and Denoising Convolutional Neural Network ForAutonomous Driving}},
  year = {2021},
  eprint = {2109.07110},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2109.07110},
}
Figure