GAN-Based Video Denoising with Attention Mechanism for Field-Applicable Pig Detection System

Bo Zhao, Othmane Atif, Jonguk Lee, Daihee Park, Yongwha ChungSensors 2022 · DOI

TL;DR

This paper introduces a novel system for robust, real-time pig detection in intensive farms using infrared cameras. It tackles the challenge of "haze spots" caused by insect secretions on camera lenses, which degrade image quality and hinder detection. The core innovation is a GAN-based denoising preprocessor with a multi-scale spatial attention mechanism that effectively removes these specific noise artifacts, significantly improving pig detection accuracy.

What Problem Does This Paper Address?

Intensive pig farming relies on continuous monitoring for animal health and welfare. Infrared cameras are ideal for this due to their non-invasive nature and ability to work in low-light conditions. However, a practical issue arises from the farm environment: insect secretions accumulate on camera lenses, creating "haze spots." These spots, caused by IR reflections, obscure parts of the image and can be mistaken for pigs or cause pigs to be missed entirely by detection algorithms. Existing image dehazing techniques aren't well-suited for these localized, unpredictable "haze spots," necessitating a specialized solution to maintain reliable pig monitoring.

Methods

The proposed system comprises three main components:

  1. Data Collector: An infrared camera captures images from the pigpen.
  2. Preprocessor: This is the core innovation. It takes noisy infrared images and transforms them into clean ones.
  3. Attention-based Denoising Model: This model is built upon a Generative Adversarial Network (GAN) and a U-Net architecture.
  4. Multi-Scale Spatial Attention Module: Embedded within the U-Net, this module allows the model to focus on areas with haze spots at different feature levels. It learns to strengthen image transformations in noisy regions while minimizing changes in clean areas, leading to more effective noise removal.
  5. Noise Simulation: Since obtaining perfectly paired real-world noisy and clean images of the same scene is difficult due to pig movement, the authors developed a noise simulation method. This method generates realistic haze spots by varying transmission rates and atmospheric light, allowing the GAN to be trained on simulated paired data.
  6. Pig Detector: A YOLOv5 model is used for the final pig detection task, chosen for its speed and accuracy.

Datasets

  • Real-world Data: Infrared images were collected from a pigpen in Gyeongsangnam-do, Korea, including clean images and images with varying levels of insect-induced haze spots (low-level and high-level noisy images).
  • Simulated Data: A crucial aspect of the training involved creating synthetic datasets. Clean images were augmented with simulated haze spots (uniform, linear-form, and exponential-form) to create paired noisy-clean image datasets. A total of 18,000 image pairs were created (14,400 for training, 3600 for testing).

Key Insights and Ideas

  • Specificity of "Haze Spots": The paper highlights that the insect-induced "haze spots" are a distinct problem from general atmospheric haze. Their localized, unpredictable nature and similarity in appearance to obstacles require a tailored approach, not just standard dehazing.
  • Attention for Targeted Denoising: The multi-scale spatial attention mechanism is key. It allows the model to "learn where to look" for noise, making the denoising process more efficient and effective for these specific artifacts. This is a significant improvement over methods that treat all image regions equally.
  • Simulated Data for Real-World Problems: The authors demonstrate the effectiveness of using simulated noise to overcome the practical challenge of acquiring perfectly paired real-world data, a common hurdle in many computer vision applications.
  • End-to-End Efficiency: The entire system, from denoising to detection, is designed to operate in real-time, making it suitable for practical, continuous monitoring in farm environments.

Key Results

  • Significant Accuracy Improvement: The proposed denoising preprocessor, when integrated into the pig detection system, improved the average precision of pig detection from 0.766 to 0.906.
  • Minimal Computational Overhead: The denoising preprocessor added only 4.8 ms of execution time per image on a PC environment, demonstrating its real-time capability.
  • Superior Denoising Performance: Compared to state-of-the-art image dehazing methods (FFA-Net, GridDehaze-Net, SpA-GAN), the proposed model showed better performance on real-noise images, particularly in terms of average precision (AP). For high-level noisy images, the proposed model achieved an AP of 0.906, outperforming others.
  • Attention Module Effectiveness: The attention module significantly boosted denoising performance, increasing PSNR from 29.1 dB to 33.3 dB on simulated noisy images.

Limitations and Future Work

  • Obstacle Interference: The model sometimes misclassifies obstacles close to the camera that cause IR reflection as noise, changing their color.
  • Distortion and Incomplete Removal: In some cases, the denoising model might distort non-noise regions or fail to completely remove noise.
  • Color Degradation: In one instance, color degradation in the denoised image led to a single pig being detected as two.
  • Future Work:
    • Improve the noise simulation method for greater realism.
  • Explore alternative denoising methods that do not require paired noisy and clean images, enabling training on unpaired real-world data.
  • Collect more diverse data to test performance across different camera locations and scenarios.

Summary updated

Abstract

Infrared cameras allow non-invasive and 24 h continuous monitoring. Thus, they are widely used in automatic pig monitoring, which is essential to maintain the profitability and sustainability of intensive pig farms. However, in practice, impurities such as insect secretions continuously pollute camera lenses. This causes problems with IR reflections, which can seriously affect pig detection performance. In this study, we propose a noise-robust, real-time pig detection system that can improve accuracy in pig farms where infrared cameras suffer from the IR reflection problem. The system consists of a data collector to collect infrared images, a preprocessor to transform noisy images into clean images, and a detector to detect pigs. The preprocessor embeds a multi-scale spatial attention module in U-net and generative adversarial network (GAN) models, enabling the model to pay more attention to the noisy area. The GAN model was trained on paired sets of clean data and data with simulated noise. It can operate in a real-time and end-to-end manner. Experimental results show that the proposed preprocessor was able to significantly improve the average precision of pig detection from 0.766 to 0.906, with an additional execution time of only 4.8 ms on a PC environment.
BibTeX
@inproceedings{zhao2022gan,
  author = {Bo Zhao and Othmane Atif and Jonguk Lee and Daihee Park and Yongwha Chung},
  title = {{GAN-Based Video Denoising with Attention Mechanism for Field-Applicable Pig Detection System}},
  booktitle = {Sensors},
  year = {2022},
  doi = {10.3390/s22103917},
}

Loading related content...

Figure