YOLOv5s-Fog: An Improved Model Based on YOLOv5s for Object Detection in Foggy Weather Scenarios

Xianglin Meng, Yi Liu, Lili Fan, Jingjing FanSensors 2023 · DOI

TL;DR

This paper introduces YOLOv5s-Fog, an enhanced YOLOv5s model specifically designed for robust object detection in foggy conditions. By integrating a Swin Transformer-based feature extraction layer (SwinFocus), a decoupled detection head, and Soft-NMS for post-processing, the model significantly improves detection accuracy for blurred and small objects in adverse weather, achieving a 5.4% increase in mAP on the RTTS dataset.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on accurate object detection to navigate safely. However, foggy weather poses a significant challenge. The scattering and absorption of light by fog particles blur object features, reduce contrast, and can even lead to objects being completely lost in the image. This degradation severely impacts the performance of standard object detection algorithms, compromising the safety and reliability of vehicles operating in such conditions. Existing methods often resort to image enhancement or dehazing as a preprocessing step, which can introduce artifacts or be computationally expensive. This paper aims to develop a direct object detection method that can effectively handle foggy scenes without relying on such preprocessing.

Methods

The YOLOv5s-Fog model builds upon the efficient YOLOv5s architecture and introduces three key improvements:

  • SwinFocus Layer: This novel feature detection layer, inspired by the Swin Transformer, is integrated into the YOLOv5s backbone. SwinFocus leverages a hierarchical feature representation and a window-based self-attention mechanism to capture subtle object features and global context more effectively, even in blurred images. It decomposes spatial and channel dimensions for better feature aggregation.
  • Decoupled Head: The traditional YOLO head, which jointly handles classification and bounding box regression, is replaced with a decoupled head. This design separates these tasks into distinct sub-branches, allowing each to specialize and better address the challenges of reduced contrast and indistinct boundaries in foggy images.
  • Soft-NMS: The standard Non-Maximum Suppression (NMS) algorithm, which can aggressively remove overlapping bounding boxes, is replaced with Soft-NMS. Soft-NMS uses a confidence decay factor to gradually reduce the scores of overlapping boxes rather than eliminating them entirely, which is crucial for handling the increased object overlap common in foggy scenes and preventing the suppression of valid detections.

Datasets

The study utilizes the following datasets:

  • Training Data: A combination of VOC2007, VOC2012, and COCO datasets (referred to as V_C_t).
  • Testing Data:
    • V_n_ts (derived from VOC2007_test) for evaluating performance on standard datasets.
  • RTTS (Real-world Thermal Target Detection Dataset) for evaluating performance specifically in foggy weather conditions. RTTS is highlighted as the only existing real-world foggy scene object detection dataset with multi-class labels.

Key Insights and Ideas

  • Direct Foggy Scene Detection is Viable: The research demonstrates that it's possible to achieve high performance in foggy conditions without relying on explicit image dehazing or enhancement preprocessing steps, which can be prone to introducing noise.
  • Swin Transformer's Power for Fog: The integration of Swin Transformer's attention mechanism (via SwinFocus) proves highly effective in capturing subtle, degraded features in foggy images, outperforming traditional convolutional approaches.
  • Task Separation Improves Robustness: Decoupling the classification and regression tasks in the detection head allows the model to better adapt to the visual impairments caused by fog, leading to more accurate localization and classification.
  • Soft-NMS is Crucial for Overlap: The increased object overlap in foggy scenes necessitates a more nuanced suppression strategy like Soft-NMS to avoid discarding valid detections.

Key Results

  • YOLOv5s-Fog achieved a mean Average Precision (mAP) of 73.4% on the RTTS dataset, representing a 5.4% increase compared to the baseline YOLOv5s model (68.00%).
  • On the V_n_ts dataset, YOLOv5s-Fog reached an mAP of 92.23%, outperforming several other state-of-the-art methods.
  • The ablation studies showed significant improvements with each added module:
  • YOLOv5s + SwinFocus: +2.15% mAP on RTTS.
  • YOLOv5s + SwinFocus + Decoupled Head: +1.64% mAP on RTTS.
  • YOLOv5s + SwinFocus + Decoupled Head + Soft-NMS: +1.61% mAP on RTTS.
  • The model demonstrates excellent detection capabilities for both blurred objects and smaller targets in dense fog.

Limitations and Future Work

  • Computational Burden: The introduction of SwinFocus increases the model's parameters and computational burden, although the authors argue it's justified by the performance gains in challenging scenarios.
  • Dataset Limitations: The paper notes the scarcity of datasets for training and testing object detection algorithms under adverse weather conditions.
  • Future Work: The authors plan to focus on constructing more comprehensive datasets for extreme weather conditions and developing even more efficient network architectures to further enhance model accuracy in such environments.

Summary updated

Abstract

In foggy weather scenarios, the scattering and absorption of light by water droplets and particulate matter cause object features in images to become blurred or lost, presenting a significant challenge for target detection in autonomous driving vehicles. To address this issue, this study proposes a foggy weather detection method based on the YOLOv5s framework, named YOLOv5s-Fog. The model enhances the feature extraction and expression capabilities of YOLOv5s by introducing a novel target detection layer called SwinFocus. Additionally, the decoupled head is incorporated into the model, and the conventional non-maximum suppression method is replaced with Soft-NMS. The experimental results demonstrate that these improvements effectively enhance the detection performance for blurry objects and small targets in foggy weather conditions. Compared to the baseline model, YOLOv5s, YOLOv5s-Fog achieves a 5.4% increase in mAP on the RTTS dataset, reaching 73.4%. This method provides technical support for rapid and accurate target detection in adverse weather conditions, such as foggy weather, for autonomous driving vehicles.
BibTeX
@inproceedings{meng2023yolov,
  author = {Xianglin Meng and Yi Liu and Lili Fan and Jingjing Fan},
  title = {{YOLOv5s-Fog: An Improved Model Based on YOLOv5s for Object Detection in Foggy Weather Scenarios}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23115321},
}
Figure