A Proactive Recognition System for Detecting Commercial Vehicle Driver's Distracted Behavior

Xintong Yan, Jie He, Guanhe Wu, Changjian Zhang, Chenwei WangSensors 2022 · DOI

TL;DR

This paper introduces a proactive system for detecting distracted driving in commercial vehicles using a cascaded Convolutional Neural Network (CNN) approach. By analyzing driver posture across various scenarios (day/night, different distractions), the system aims to improve safety by providing real-time warnings, outperforming traditional methods in both accuracy and speed.

What Problem Does This Paper Address?

Road traffic accidents, particularly those involving commercial vehicles, are a significant cause of death and economic loss. A large proportion of these accidents are attributed to driver error, with distracted driving being a major contributor. Existing driver monitoring systems often have limitations: they may focus on a narrow range of distractions, struggle with varying light conditions (day vs. night), or lack the comprehensive analysis needed for complex behaviors. This paper addresses the need for a more robust and proactive system that can identify a wider array of distracted behaviors in commercial vehicle drivers, especially under challenging conditions like nighttime driving.

Methods

The core of this research is a cascaded Convolutional Neural Network (CNN) framework. Instead of a single, monolithic model, the authors propose breaking down the complex task of driver posture recognition into several specialized sub-models. These sub-models are designed based on the analysis of driver posture characteristics, including continuity, diversity, superposition, similarity, transitional states, and mutual influence (compatibility/incompatibility) between different postures.

Key methodological aspects include:

  • Multi-task Classification: The system is designed to handle multiple, potentially overlapping, driver postures simultaneously.
  • Cascaded Architecture: This is the novel aspect. The CNN is structured in stages. For instance, if a "two-hand driving" posture is detected, subsequent models might be bypassed, improving efficiency. This leverages the compatibility and mutual exclusion of different postures identified in the study.
  • Sub-Model Specialization: Five sub-models are developed:
  • Hand Posture Set (two-hand, one-hand, no-hand driving)
    • Not Looking Ahead
    • Smoking
    • Calling the Phone
    • Behavior Posture Set (controlling gear, using phone/rack, dashboard, eating/drinking, taking things)
  • Orthogonalization Method: Used for label processing to simplify the representation of superimposed postures, reducing redundancy and improving efficiency.
  • Day and Night Models: Separate models are trained and evaluated for daytime and nighttime conditions due to significant differences in lighting and image capture techniques.

Datasets

The study utilized a large dataset collected from commercial vehicle drivers:

  • Total Images: 666,060 images (452,840 daytime, 213,220 nighttime).
  • Data Sources: Images were collected from Top Chains International Logistics Company (TCILC) and Hurricane Logistics Company (HLC).
  • Dataset Types:
    • Naturalistic Driving Data: Images captured during actual driving.
  • Supplementary Driving Data: Images created by reproducing dangerous postures to ensure a balanced dataset and improve the model's ability to detect less frequent but critical behaviors.
  • Image Resolution: Two resolutions were used for CNN training: 180x320 and 360x640.

Key Insights and Ideas

  • Posture Characteristics Matter: Understanding the dynamic and complex nature of driver postures (continuity, diversity, superposition) is crucial for building effective recognition systems.
  • Cascading for Efficiency: A cascaded approach, where models are chained and can bypass subsequent stages based on initial findings, significantly improves both recognition speed and resource efficiency.
  • Day vs. Night Performance Trade-offs: Night models are faster but slightly less accurate than daytime models, likely due to image quality limitations and smaller dataset sizes.
  • Superposition is Common: Drivers often exhibit multiple distracted behaviors simultaneously, requiring models capable of handling such complex states.
  • Commercial Vehicle Specificity: Research on commercial vehicle drivers is less abundant than for passenger cars, despite their higher risk profile.

Key Results

  • Overall Accuracy: The cascaded CNN model achieved a daytime accuracy of 98.68%, an improvement of 0.85% over the non-cascaded model (97.83%). For night data, the cascaded model achieved 97.48% accuracy.
  • Speed Improvement: The cascaded model was 10.4% faster than the non-cascaded model during daytime testing (405 ms vs. 452 ms). Nighttime speed was also significantly faster (403 ms vs. 362 ms for cascaded vs. non-cascaded).
  • Sub-Model Performance: Individual sub-models demonstrated high accuracy, with the Hand Posture Set achieving up to 99.3% (daytime) and the Behavior Posture Set reaching 99.35% (daytime) and 99.77% (nighttime).
  • Night Model Performance: Night models generally showed slightly lower accuracy but better speed compared to their daytime counterparts.

Limitations and Future Work

  • Limitations:
    • The study acknowledges that the accuracy of night models is slightly lower, potentially due to the immaturity of night image capture techniques and smaller sample sizes.
  • The research primarily focuses on posture recognition, and while it aims to detect distracted behavior, further work could explore dynamic changes in dangerous postures and evaluate different danger levels.
  • Future Work:
    • Develop policies and countermeasures to improve driver safety.
  • Provide information for designing real-time driver monitoring and warning systems, as well as automatic driving systems.
  • Investigate the dynamic changes of dangerous postures and their danger levels.
  • Establish or open more datasets with diverse scenes to further improve models.
  • Explore distracted behaviors like texting or excessive speeding.
  • Integrate vehicle state parameters with driver posture for a more comprehensive evaluation system.

Summary updated

BibTeX
@article{yan2022proactive,
  author = {Xintong Yan and Jie He and Guanhe Wu and Changjian Zhang and Chenwei Wang},
  title = {{A Proactive Recognition System for Detecting Commercial Vehicle Driver's Distracted Behavior}},
  journal = {Sensors},
  year = {2022},
  doi = {10.3390/s22062373},
}
Figure