Automatic Detection Method for Black Smoke Vehicles Considering Motion Shadows

Han Wang, Ke Chen, Yanfeng LiSensors 2023 · DOI

TL;DR

This paper tackles the challenge of accurately detecting vehicles emitting black smoke, a major air pollutant, by introducing a novel "segmentation-classification" approach. By explicitly considering and segmenting motion shadows (which can be mistaken for smoke), their Y-MobileNetv3 model achieves a significant accuracy improvement and maintains real-time performance.

What Problem Does This Paper Address?

Vehicle emissions are a primary source of air pollution. Detecting vehicles that emit excessive black smoke is crucial for environmental monitoring and enforcement. However, existing detection methods struggle with interference from "motion shadows" – the dark areas cast by vehicles under sunlight – which visually resemble black smoke. This can lead to misclassifications (false positives and false negatives), reducing the accuracy and reliability of automatic detection systems. Traditional approaches often try to remove shadows first, which is complex and can alter important features. This paper aims to overcome this by directly addressing the coexistence of smoke and shadows.

Methods

The proposed method employs a two-stage approach:

  1. Object Localization (YOLOv5s): The YOLOv5s object detection model is used to identify and extract regions containing moving objects. Crucially, this includes not only motor vehicles but also potential black smoke emissions and motion shadows. This initial step effectively isolates relevant areas from the surveillance video.
  2. Segmentation and Classification (Superpixel + MobileNetv3):
    • Superpixel Segmentation (SLIC): The extracted motion object regions are then processed using the Simple Linear Iterative Clustering (SLIC) algorithm. SLIC groups similar pixels into "superpixels," creating segmented images. This step is key to differentiating between black smoke emissions and motion shadows based on their visual characteristics. The optimal parameters for SLIC (k=1500, m=10) were determined through experimental analysis.
  3. Classification (MobileNetv3): These superpixel images are fed into a lightweight MobileNetv3 network. This network is trained to classify the segmented regions into "black smoke exhaust" and "motion shadows" (or "motor vehicles" if no smoke is present). This "segmentation-classification" strategy allows the model to learn to distinguish between the two phenomena directly, rather than trying to remove shadows first.

Datasets

The study utilizes road traffic surveillance videos as the source for their dataset.

  • Initially, 2900 images containing motor vehicles were extracted.
  • For training the Y-MobileNetv3 model (considering motion shadows), the dataset comprised 1082 images of black smoke emissions, 1035 images of motion shadows, and 1118 images of motor vehicles, all processed via superpixel segmentation.
  • For training the N-MobileNetv3 model (not considering motion shadows), the dataset used 2320 non-black smoke emissions and 580 black smoke emissions.

The paper does not mention the use of publicly available benchmark datasets for training or evaluation, suggesting a custom dataset derived from their collected videos.

Key Insights and Ideas

  • Shadows as a Feature, Not a Nuisance: The core innovation is treating motion shadows as a distinct category to be segmented and classified, rather than an artifact to be removed. This reframes the problem from "detect smoke, then remove shadows" to "segment smoke and shadows, then classify smoke."
  • Superpixel Segmentation for Fine-Grained Distinction: SLIC proves effective in grouping pixels with similar features, enabling the model to discern subtle differences between the hazy appearance of smoke and the more textured appearance of shadows.
  • Lightweight Model for Real-Time Deployment: The choice of MobileNetv3 is crucial for achieving real-time detection, a critical requirement for surveillance applications, without sacrificing significant accuracy.
  • "Segmentation-Classification" Advantage: This approach directly addresses the visual similarity between smoke and shadows, leading to improved accuracy and reduced false positives/negatives compared to models that ignore this interference.

Key Results

  • Accuracy Improvement: The Y-MobileNetv3 model, which accounts for motion shadows, achieved an average accuracy of 95.17%. This is a significant 4.73% improvement over the N-MobileNetv3 model (90.34%) which does not consider motion shadows.
  • Precision and Recall: Y-MobileNetv3 also showed superior precision (96.03%) and recall (94.77%) compared to N-MobileNetv3 (91.39% and 90.19%, respectively).
  • Real-Time Performance: The model achieves an average inference time of 7.3 ms per image, meeting real-time detection requirements. This is slightly faster than the N-MobileNetv3 model (8.7 ms).
  • Reduced False Positives/Negatives: The paper highlights that the proposed method effectively reduces false positives and negatives, particularly in cases of trace black smoke emissions.

Limitations and Future Work

  • Limitations:
    • The study relies on a custom dataset derived from road traffic surveillance videos, and the generalizability to diverse environmental conditions or different types of vehicles might need further validation.
  • While the paper mentions the challenge of quantitative calculation of black smoke exhaust concentration, their current approach primarily focuses on detection and classification.
  • Future Work:
    • The authors suggest further research into more in-depth analysis of image segmentation for moving targets to better differentiate between black smoke exhaust and motion shadows.
  • They also propose exploring a color-coded approach to depict the outline of black smoke exhaust.
  • The ultimate goal is to enable hierarchical classification management of smoky vehicles and assist law enforcement in monitoring them more efficiently.

Summary updated

Abstract

Various statistical data indicate that mobile source pollutants have become a significant contributor to atmospheric environmental pollution, with vehicle tailpipe emissions being the primary contributor to these mobile source pollutants. The motion shadow generated by motor vehicles bears a visual resemblance to emitted black smoke, making this study primarily focused on the interference of motion shadows in the detection of black smoke vehicles. Initially, the YOLOv5s model is used to locate moving objects, including motor vehicles, motion shadows, and black smoke emissions. The extracted images of these moving objects are then processed using simple linear iterative clustering to obtain superpixel images of the three categories for model training. Finally, these superpixel images are fed into a lightweight MobileNetv3 network to build a black smoke vehicle detection model for recognition and classification. This study breaks away from the traditional approach of "detection first, then removal" to overcome shadow interference and instead employs a "segmentation-classification" approach, ingeniously addressing the coexistence of motion shadows and black smoke emissions. Experimental results show that the Y-MobileNetv3 model, which takes motion shadows into account, achieves an accuracy rate of 95.17%, a 4.73% improvement compared with the N-MobileNetv3 model (which does not consider motion shadows). Moreover, the average single-image inference time is only 7.3 ms. The superpixel segmentation algorithm effectively clusters similar pixels, facilitating the detection of trace amounts of black smoke emissions from motor vehicles. The Y-MobileNetv3 model not only improves the accuracy of black smoke vehicle recognition but also meets the real-time detection requirements.
BibTeX
@inproceedings{wang2023automatic,
  author = {Han Wang and Ke Chen and Yanfeng Li},
  title = {{Automatic Detection Method for Black Smoke Vehicles Considering Motion Shadows}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23198281},
}
Figure