DriveGazen: Event-Based Driving Status Recognition Using Conventional Camera

Xiaoyin Yang, Xin YangAAAI 2025 · DOI

TL;DR

This paper introduces DriveGazen, a novel system for recognizing driver states (like fatigue or distraction) using a conventional camera. It achieves this by generating "event frames" from standard video and processing them with a specialized spiking neural network that incorporates spatial cues from the original video, outperforming existing methods.

What Problem Does This Paper Address?

Ensuring traffic safety hinges on understanding a driver's state – are they attentive, drowsy, or distracted? While RGB cameras offer rich information, accurately inferring these states is challenging due to varying lighting, head movements, and partial occlusions. Existing methods often struggle with temporal nuances, especially in low-light conditions. The authors aim to bridge this gap by developing a robust, low-cost system that can reliably detect driver states from eye movements, even under difficult conditions, by leveraging event-based sensing principles with conventional cameras.

Methods

The core of DriveGazen lies in its innovative approach to event frame generation and processing:

  • Video to Event (v2e) Generation: Standard RGB video frames are converted into grayscale and then processed to generate synthetic Dynamic Vision Sensor (DVS) events. This process mimics how event cameras work by detecting changes in pixel intensity over time, effectively capturing temporal information that is often lost in traditional frame-based analysis. The v2e method accounts for factors like quantization noise, analog bandwidth limitations, and spontaneous "leak" events.
  • Attention Driving State Network (ADSN): This is the custom-built neural network designed to process the event frames and original intensity frames. ADSN comprises:
  • Spatial Feature Extractor: This module extracts spatial information from the first and last frames of the input video. It uses multi-scale self-attention to capture both local and global spatial cues.
  • Temporal Feature Extractor: This is a spiking neural network (SNN) that processes the generated event frames. SNNs are biologically inspired and excel at handling temporal information through their "spiking" mechanism, which mimics neuronal firing. This module is designed to learn temporal dependencies of arbitrary length.
  • Guide Attention Module: This crucial component fuses the spatial and temporal information. It uses spatial features to guide the learning and enhancement of event frames, ensuring that important spatial cues inform the temporal analysis. This module injects texture information into the event-based processing, which is often lacking in raw event data.

Datasets

  • DriveGaze Dataset: The authors introduce a new, publicly available dataset specifically for eye-based driving state recognition. It comprises data from 47 volunteers of diverse ages, genders, and races, captured using a wearable DG3 eye tracker. This dataset includes both original video frames and corresponding event frames, covering seven driving states.
  • Single-eye Event-based Emotion (SEE) Dataset: DriveGazen was also validated on this existing dataset to demonstrate its generalizability and superiority on emotion recognition tasks.

Key Insights and Ideas

  • Bridging the Gap between Event and Conventional Cameras: The paper successfully demonstrates that generating synthetic event frames from conventional cameras can effectively leverage the strengths of both worlds – the rich spatial information of RGB cameras and the temporal sensitivity of event cameras.
  • The Power of Spiking Neural Networks for Temporal Dynamics: The use of SNNs in the temporal feature extractor proves highly effective for capturing subtle, time-varying driver states that might be missed by traditional recurrent networks.
  • Spatial Cues are Crucial for Event-Based Temporal Analysis: The "Guide Attention Module" highlights that simply processing events isn't enough. Injecting spatial context from the original frames significantly enhances the accuracy and robustness of the temporal analysis.
  • Low-Cost Wearable Solution: The development of a prototype using a conventional camera integrated into glasses offers a practical and affordable path towards widespread adoption of driver monitoring systems.

Key Results

  • On the newly introduced DriveGaze dataset, DriveGazen achieved state-of-the-art performance, with the E8-S7 configuration yielding the best results at 92.4% WAR and UAR.
  • On the SEE dataset, DriveGazen (E4-S3) outperformed the runner-up method (SEEN) by 1% in WAR and UAR for emotion recognition.
  • The method showed significant robustness, outperforming SEEN by at least 4% in accuracy under overexposure and HDR lighting conditions.
  • Ablation studies confirmed that both the first and last intensity frames are essential, and all components of ADSN contribute to the overall performance.

Limitations and Future Work

  • Reliance on Synthetic Events: While effective, the method's performance is partially dependent on the quality of the synthetic event generation. Performance degradation might occur when intensity variations are minimal.
  • Future Research Directions: The authors suggest further improving robustness as an exciting direction for future research. This could involve exploring more advanced event generation techniques or network architectures that are even more resilient to challenging conditions.

Summary updated

BibTeX
@inproceedings{yang2025drivegazen,
  author = {Xiaoyin Yang and Xin Yang},
  title = {{DriveGazen: Event-Based Driving Status Recognition Using Conventional Camera}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.33005},
}
Figure