Research on Driving Obstacle Detection Technology in Foggy Weather Based on GCANet and Feature Fusion Training

Zhaohui Liu, Shiji Zhao, Xiao WangSensors 2023 · DOI

TL;DR

This paper tackles the critical challenge of detecting driving obstacles in foggy weather by proposing a novel method that combines the GCANet image defogging algorithm with a feature fusion training approach. By intelligently fusing preserved edge features from defogged images with convolutional features, the method significantly boosts obstacle detection accuracy and recall, crucial for safe autonomous driving.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on accurate environment perception to ensure safety. However, adverse weather conditions, particularly fog, severely degrade the quality of visual sensor data, leading to challenges in detecting obstacles. Traditional approaches often fall into two categories: first, defogging followed by detection, which can lead to information loss (especially of edge details) after defogging; or second, direct detection without defogging, which suffers from low accuracy due to visual occlusion. This paper aims to bridge this gap by developing a method that effectively leverages the strengths of both defogging and feature fusion to overcome the limitations of existing techniques in foggy environments.

Methods

The core of this research lies in a two-pronged approach:

  1. GCANet for Defogging: The authors select the GCANet (Gated Context Aggregation Network) algorithm for image defogging. GCANet is chosen for its ability to preserve important image information, particularly edge features, without relying on prior knowledge, thus mitigating the information loss issue common in other defogging methods.
  2. Edge and Convolutional Feature Fusion Training: Building upon the defogged images, the paper proposes a novel training strategy for the YOLOv5 object detection network. This strategy involves:
  3. Edge Feature Extraction: Extracting edge features from clear-day images using the Sobel operator.
  4. Convolutional Feature Extraction: Utilizing the YOLOv5 backbone (specifically Focus and CSP1_X structures) to extract convolutional features from clear-day images.
  5. Feature Fusion: Combining the extracted edge features and convolutional features. This fusion is achieved through operations like "add" (to superimpose features and increase information content) and "Concat" (to splice feature maps and fuse multi-scale information). This fused representation is then used to train the YOLOv5 model.

The rationale is that while fog degrades overall image quality, GCANet effectively preserves edge information, which is vital for object contour and localization. By fusing these preserved edge features with the rich semantic information from convolutional features, the model becomes more robust to foggy conditions.

Datasets

The study primarily utilizes the KITTI dataset for training and validation. Specifically, clear-day images from KITTI are used to train the feature fusion model. For testing, a synthesized foggy image test set is created by applying a fog simulation model (based on Koschmieder's law) to clear-day images from KITTI and the BDD100K dataset.

Key Insights and Ideas

  • Edge Features are Resilient: The research highlights that even after defogging, edge features of objects in foggy images are relatively well-preserved compared to color and grayscale information. This resilience makes them a valuable asset for improving detection accuracy.
  • GCANet's Advantage: GCANet proves to be a superior defogging choice for this task because it retains more image information, especially edge details, compared to other methods like DCP and AOD-Net, which is crucial for the subsequent feature fusion.
  • Fusion is Key: Simply defogging isn't enough. The true innovation lies in the fusion of these preserved edge features with standard convolutional features. This synergistic approach allows the model to leverage both detailed structural information and semantic understanding.
  • YOLOv5 as a Robust Backbone: The choice of YOLOv5 as the detection architecture, with its Focus and CSP1_X structures, provides an efficient and effective backbone for feature extraction and subsequent fusion.

Key Results

The proposed method demonstrates significant improvements in obstacle detection performance in foggy weather:

  • mAP Improvement: The method achieves a substantial increase in mean Average Precision ([email protected]) of 45% compared to conventional training using only the KITTI dataset.
  • Recall Improvement: Recall is improved by 19% compared to conventional training.
  • Comparison with GCANet + Conventional Training: Compared to using GCANet defogging followed by conventional YOLOv5 training, the proposed fusion method improves recall by 12% and mAP by 9%.
  • Overall Performance: The final proposed method achieves a [email protected] of 0.928, with precision and recall of 0.927 and 0.881, respectively. This performance is superior to other tested combinations.
  • Time Efficiency: The detection time per image remains in the milliseconds range, ensuring real-time performance, which is critical for autonomous driving.

Limitations and Future Work

  • Adverse Weather Verification: The authors acknowledge that the experimental results have only been verified for foggy weather. They suggest that future work should include verifying the method's effectiveness in other adverse weather conditions, such as rainy days.
  • Precision Comparison: The paper notes that the precision of their proposed method is slightly lower than that of the DCP + edge and convolutional feature fusion training method. Future work will focus on improving this precision index.
  • Further Improvement: The authors aim to further improve the proposed method's performance, particularly in rainy weather scenarios.

Summary updated

Abstract

The issues of the degradation of the visual sensor's image quality in foggy weather and the loss of information after defogging have brought great challenges to obstacle detection during autonomous driving. Therefore, this paper proposes a method for detecting driving obstacles in foggy weather. The driving obstacle detection in foggy weather was realized by combining the GCANet defogging algorithm with the detection algorithm-based edge and convolution feature fusion training, with a full consideration of the reasonable matching between the defogging algorithm and the detection algorithm on the basis of the characteristics of obvious target edge features after GCANet defogging. Based on the YOLOv5 network, the obstacle detection model is trained using clear day images and corresponding edge feature images to realize the fusion of edge features and convolution features, and to detect driving obstacles in a foggy traffic environment. Compared with the conventional training method, the method improves the mAP by 12% and recall by 9%. In contrast to conventional detection methods, this method can better identify the image edge information after defogging, which significantly enhances detection accuracy while ensuring time efficiency. This is of great practical significance for improving the safe perception of driving obstacles under adverse weather conditions, ensuring the safety of autonomous driving.
BibTeX
@inproceedings{liu2023research,
  author = {Zhaohui Liu and Shiji Zhao and Xiao Wang},
  title = {{Research on Driving Obstacle Detection Technology in Foggy Weather Based on GCANet and Feature Fusion Training}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23052822},
}
Figure