An Effective Image Denoising Method for UAV Images via Improved Generative Adversarial Networks

Ruihua Wang, Xiongwu Xiao, Bingxuan Guo, Qianqing Qin, Ruizhi Chen2018 · DOI

TL;DR

This paper introduces a novel deep learning approach, DUGAN, for denoising Unmanned Aerial Vehicle (UAV) images. By combining generative adversarial networks (GANs) with a perceptual reconstruction loss, DUGAN effectively removes noise while preserving crucial ground object details and textures, outperforming existing methods in quantitative and qualitative evaluations.

What Problem Does This Paper Address?

Unmanned Aerial Vehicles (UAVs) are increasingly used for collecting high-resolution aerial imagery. However, these images are often plagued by noise introduced by various environmental factors like lighting changes, electromagnetic interference, and the UAV's own mechanical vibrations. This noise degrades image quality, obscuring important details and textures of ground objects, which is critical for applications like remote sensing, surveillance, and mapping. Existing denoising methods often struggle to balance noise reduction with the preservation of fine details, leading to overly smooth images or residual noise.

Methods

The core of this work is the Denoise UAV image Generative Adversarial Network (DUGAN), a deep learning architecture built upon Generative Adversarial Networks (GANs). DUGAN consists of two main components:

  • Generative Model (G): This model is designed to learn the mapping from noisy UAV images to clean, denoised images. It features a deep architecture with 14 residual blocks. Residual blocks are crucial for enabling deeper networks by facilitating gradient flow and preventing vanishing gradients, which is essential for learning complex image features. Batch normalization is also employed within these blocks to stabilize training and improve convergence.
  • Discriminative Model (D): This model acts as a critic, tasked with distinguishing between real clean UAV images and the images generated by the generative model. This adversarial process drives the generative model to produce increasingly realistic and high-quality denoised images.

A key innovation is the loss function used to train DUGAN. It's a combination of three components:

  • Perceptual Reconstruction Loss ($L_{pe}$): This loss is calculated using features extracted from a pre-trained VGG network. Instead of just comparing pixel values, it minimizes the differences in high-level features, encouraging the denoised images to retain perceptual similarity to the ground truth, particularly in terms of textures and structures.
  • Generative Adversarial Loss ($L_{ga}$): This is the standard GAN loss that encourages the generator to fool the discriminator.
  • Pixel-based Euclidean Loss ($L_{pi}$): This is a traditional loss that measures the pixel-wise difference between the denoised image and the ground truth, helping to recover fine details.

The combination of these losses, with carefully chosen weights, allows DUGAN to effectively denoise while preserving critical image content.

Datasets

The paper introduces a new UAV image dataset for training and testing.

  • Training Data: A set of 400 clean UAV images (360x360 pixels) were created by cropping larger images. Noise was then synthetically added to these clean images to create noisy training pairs.
  • Testing Data:
    • 40 UAV images were used for testing.
  • An additional set of 100 smaller UAV images (cars and trucks) was used for classification experiments.

The testing data was collected from various locations (Guiyang, Yangjiang, Wuhan) using different UAVs and cameras, providing a diverse set of real-world UAV imagery.

Key Insights and Ideas

  • Perceptual Loss for UAV Images: The paper highlights that traditional pixel-wise losses often lead to overly smooth images, which is detrimental for UAV imagery where fine details and textures are paramount. Using perceptual loss, derived from deep features, is crucial for preserving these important characteristics.
  • GANs for Realistic Denoising: GANs provide a powerful framework for generating realistic images. By framing denoising as a generative task, the model learns to produce outputs that are not only noise-free but also visually plausible and structurally similar to real, clean images.
  • Deep Residual Networks are Key: The success of DUGAN hinges on its deep architecture, enabled by residual blocks. This allows the network to learn complex mappings and capture subtle image features that are often lost in shallower networks.
  • Combined Loss for Optimal Results: The synergy between perceptual loss, adversarial loss, and pixel loss is critical. Perceptual loss guides the overall structure and texture, adversarial loss ensures realism, and pixel loss helps recover fine details.

Key Results

  • Visual Quality: DUGAN-denoised images exhibit clearer ground object edges and more detailed textures compared to images denoised by other methods. They are visually more appealing and closer to the original clean images.
  • Quantitative Metrics: DUGAN consistently achieved the highest PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) scores across different noise levels (20, 35, and 55) when compared to methods [8], [5], and [17]. For instance, at noise level 55, the average PSNR was 26.6576, and the average SSIM was 0.7689, outperforming the other methods.
  • Image Matching: SIFT matching experiments showed that DUGAN produced denoised images with a significantly higher number of correct matching pairs with the original clean images, indicating better preservation of local features and textures.
  • Image Classification: DUGAN's denoised images achieved a higher classification accuracy rate (85% for testing images) when used to train a classification network, suggesting that the denoised images are more similar to real clean images.

Limitations and Future Work

  • Simulating Real UAV Noise: The paper acknowledges that the noise used in experiments was synthetically added. Future work will focus on exploring how to simulate real UAV noise more accurately to achieve even more ideal denoising effects.
  • Adversarial Learning: The authors plan to further explore adversarial learning techniques for UAV image denoising.
  • Dataset Limitations: While a new dataset was created, the paper notes the general lack of large-scale UAV image datasets for denoising tasks.

The authors believe that the deep networks and the carefully designed loss function allow for the extraction of deeper ground feature information, which is highly beneficial for subsequent UAV image applications.

Summary updated

BibTeX
@misc{wang2018effective,
  author = {Ruihua Wang and Xiongwu Xiao and Bingxuan Guo and Qianqing Qin and Ruizhi Chen},
  title = {{An Effective Image Denoising Method for UAV Images via Improved Generative Adversarial Networks}},
  year = {2018},
  doi = {10.3390/s18071985},
}
Figure