TL;DR
This paper introduces YOLOv5s-Fog, an enhanced YOLOv5s model specifically designed for robust object detection in foggy conditions. By integrating a Swin Transformer-based feature extraction layer (SwinFocus), a decoupled detection head, and Soft-NMS for post-processing, the model significantly improves detection accuracy for blurred and small objects in adverse weather, achieving a 5.4% increase in mAP on the RTTS dataset.
What Problem Does This Paper Address?
Autonomous driving systems rely heavily on accurate object detection to navigate safely. However, foggy weather poses a significant challenge. The scattering and absorption of light by fog particles blur object features, reduce contrast, and can even lead to objects being completely lost in the image. This degradation severely impacts the performance of standard object detection algorithms, compromising the safety and reliability of vehicles operating in such conditions. Existing methods often resort to image enhancement or dehazing as a preprocessing step, which can introduce artifacts or be computationally expensive. This paper aims to develop a direct object detection method that can effectively handle foggy scenes without relying on such preprocessing.
Methods
The YOLOv5s-Fog model builds upon the efficient YOLOv5s architecture and introduces three key improvements:
- SwinFocus Layer: This novel feature detection layer, inspired by the Swin Transformer, is integrated into the YOLOv5s backbone. SwinFocus leverages a hierarchical feature representation and a window-based self-attention mechanism to capture subtle object features and global context more effectively, even in blurred images. It decomposes spatial and channel dimensions for better feature aggregation.
- Decoupled Head: The traditional YOLO head, which jointly handles classification and bounding box regression, is replaced with a decoupled head. This design separates these tasks into distinct sub-branches, allowing each to specialize and better address the challenges of reduced contrast and indistinct boundaries in foggy images.
- Soft-NMS: The standard Non-Maximum Suppression (NMS) algorithm, which can aggressively remove overlapping bounding boxes, is replaced with Soft-NMS. Soft-NMS uses a confidence decay factor to gradually reduce the scores of overlapping boxes rather than eliminating them entirely, which is crucial for handling the increased object overlap common in foggy scenes and preventing the suppression of valid detections.
Datasets
The study utilizes the following datasets:
- Training Data: A combination of VOC2007, VOC2012, and COCO datasets (referred to as V_C_t).
- Testing Data:
- V_n_ts (derived from VOC2007_test) for evaluating performance on standard datasets.
- RTTS (Real-world Thermal Target Detection Dataset) for evaluating performance specifically in foggy weather conditions. RTTS is highlighted as the only existing real-world foggy scene object detection dataset with multi-class labels.
Key Insights and Ideas
- Direct Foggy Scene Detection is Viable: The research demonstrates that it's possible to achieve high performance in foggy conditions without relying on explicit image dehazing or enhancement preprocessing steps, which can be prone to introducing noise.
- Swin Transformer's Power for Fog: The integration of Swin Transformer's attention mechanism (via SwinFocus) proves highly effective in capturing subtle, degraded features in foggy images, outperforming traditional convolutional approaches.
- Task Separation Improves Robustness: Decoupling the classification and regression tasks in the detection head allows the model to better adapt to the visual impairments caused by fog, leading to more accurate localization and classification.
- Soft-NMS is Crucial for Overlap: The increased object overlap in foggy scenes necessitates a more nuanced suppression strategy like Soft-NMS to avoid discarding valid detections.
Key Results
- YOLOv5s-Fog achieved a mean Average Precision (mAP) of 73.4% on the RTTS dataset, representing a 5.4% increase compared to the baseline YOLOv5s model (68.00%).
- On the V_n_ts dataset, YOLOv5s-Fog reached an mAP of 92.23%, outperforming several other state-of-the-art methods.
- The ablation studies showed significant improvements with each added module:
- YOLOv5s + SwinFocus: +2.15% mAP on RTTS.
- YOLOv5s + SwinFocus + Decoupled Head: +1.64% mAP on RTTS.
- YOLOv5s + SwinFocus + Decoupled Head + Soft-NMS: +1.61% mAP on RTTS.
- The model demonstrates excellent detection capabilities for both blurred objects and smaller targets in dense fog.
Limitations and Future Work
- Computational Burden: The introduction of SwinFocus increases the model's parameters and computational burden, although the authors argue it's justified by the performance gains in challenging scenarios.
- Dataset Limitations: The paper notes the scarcity of datasets for training and testing object detection algorithms under adverse weather conditions.
- Future Work: The authors plan to focus on constructing more comprehensive datasets for extreme weather conditions and developing even more efficient network architectures to further enhance model accuracy in such environments.
Summary updated