Learning to Estimate Driver Drowsiness from Car Acceleration Sensors using Weakly Labeled Data

Takayuki Katsuki, Kun Zhao, Takayuki Yoshizumi

Abstract

This paper addresses the learning task of estimating driver drowsiness from the signals of car acceleration sensors. Since even drivers themselves cannot perceive their own drowsiness in a timely manner unless they use burdensome invasive sensors, obtaining labeled training data for each timestamp is not a realistic goal. To deal with this difficulty, we formulate the task as a weakly supervised learning. We only need to add labels for each complete trip, not for every timestamp independently. By assuming that some aspects of driver drowsiness increase over time due to tiredness, we formulate an algorithm that can learn from such weakly labeled data. We derive a scalable stochastic optimization method as a way of implementing the algorithm. Numerical experiments on real driving datasets demonstrate the advantages of our algorithm against baseline methods.

Introduction

Driver drowsiness is a leading cause of severe accidents, yet detecting it reliably in real-world settings remains challenging. Traditional solutions often rely on invasive sensors (EEG) or privacy-intrusive cameras, which are expensive and burdensome for drivers. Alternatively, unsupervised anomaly detection methods often fail to distinguish between drowsiness-related patterns and external factors like road conditions or weather.

The primary bottleneck is the lack of granular, timestamped ground truth data. Drivers cannot accurately self-report drowsiness levels second-by-second. This paper proposes a weakly supervised learning framework that utilizes trip-level labels—where a driver simply indicates if they felt drowsy during a specific drive—to train a high-performing estimation model using only non-intrusive car acceleration sensors.

Methodology: Weakly Supervised Learning from Temporal Progression

The proposed approach is built on two fundamental assumptions:

  1. Ordinality: Drowsiness is a continuous scale rather than a binary state.
  2. Temporal Progression: Within a trip labeled as "drowsy," a driver's tiredness generally increases monotonically over time.
Conceptual framework: Learning a drowsiness function f that increases with driving time.

Instead of standard classification, the authors employ a pair-wise loss function. For any two timestamps $t$ and $u$ in a drowsy trip, if $t > u$, the model is penalized if the estimated drowsiness score at $t$ is lower than at $u$. To handle the computational complexity of comparing all pairs, a scalable stochastic optimization method was developed, randomly sampling pairs during training to ensure efficiency.

Experimental Setup

The model was evaluated using a real-world dataset comprising 11 drowsy trips and 94 normal trips, totaling approximately 40 driving hours and 100,000 samples. Data sources included 3-axis acceleration, speed, and direction sampled at 1 Hz. Features were derived from raw sensor data, including "jerk" (the derivative of acceleration).

Results and Discussion

The proposed method significantly outperformed both unsupervised anomaly detection (Lasso, Glasso) and standard supervised classification (SVM, MLP) baselines.

Performance comparison (AUC) of the proposed method against anomaly detection and classification baselines.

Key findings include:

  • Superior Accuracy: The method achieved a trip-level AUC of 0.82, surpassing the best neural network baseline (MLP at 0.79).
  • Granular Performance: Even at the sample level, the method maintained an AUC of 0.69, while many baselines performed near or below random guessing (0.5) due to the difficulty of the weakly labeled environment.
  • Feature Importance: "X-jerk" (longitudinal acceleration changes) was identified as the most critical feature, reflecting the difficulty drowsy drivers have in maintaining smooth pedal control.
  • Efficiency: The model is approximately 30 times faster and has 100 times fewer parameters than the MLP baseline, making it ideal for edge deployment on IoT devices.

Conclusion

By leveraging the "arrow of time" as a proxy for ground truth, this research provides a practical, non-intrusive way to monitor driver safety. While the current study used a linear model and a relatively small set of drowsy trips, the framework's success with weak labels opens new avenues for large-scale safety monitoring without the need for invasive hardware. Future work will explore non-linear functions and alternative loss functions to further refine estimation accuracy.

Summary updated

Abstract

This paper addresses the learning task of estimating driver drowsiness from the signals of car acceleration sensors. Since even drivers themselves cannot perceive their own drowsiness in a timely manner unless they use burdensome invasive sensors, obtaining labeled training data for each timestamp is not a realistic goal. To deal with this difficulty, we formulate the task as a weakly supervised learning. We only need to add labels for each complete trip, not for every timestamp independently. By assuming that some aspects of driver drowsiness increase over time due to tiredness, we formulate an algorithm that can learn from such weakly labeled data. We derive a scalable stochastic optimization method as a way of implementing the algorithm. Numerical experiments on real driving datasets demonstrate the advantages of our algorithm against baseline methods.
BibTeX
@inproceedings{katsuki2020learning,
  author = {Takayuki Katsuki and Kun Zhao and Takayuki Yoshizumi},
  title = {{Learning to Estimate Driver Drowsiness from Car Acceleration Sensors using Weakly Labeled Data}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2020},
  doi = {10.1109/icassp40776.2020.9053100},
}
Figure