OpenDriver: an open-road driver state detection dataset

Delong Liu, Shichao LiSSRN 2023 · DOI · OpenReview

TL;DR

This paper introduces OpenDriver, a large-scale, multimodal dataset for driver state detection collected in real-world driving scenarios. It addresses limitations of existing datasets by offering non-intrusive physiological signal collection and extensive data, enabling new benchmarks for ECG quality assessment, biometric identification, and physiological signal analysis in complex driving environments.

What Problem Does This Paper Address?

Road traffic accidents remain a significant global issue, with driver states like fatigue, sleepiness, and distraction being major contributing factors. Wearable physiological measurements offer a promising avenue for real-time driver state monitoring, but existing datasets for this purpose suffer from several drawbacks: they are often collected in simulated environments, have poor signal quality, small sample sizes, and short data collection periods. This limits the development and validation of robust driver state detection systems for real-world driving. The paper aims to bridge this gap by introducing a comprehensive, real-world dataset and associated benchmarks.

Methods

The paper introduces the OpenDriver dataset, which collects two types of signals:

  • Six-axis motion data from the steering wheel (three-axis acceleration and three-axis angular velocity) using IMU sensors.
  • Electrocardiogram (ECG) signals collected non-intrusively via electrodes on the steering wheel cover.

Based on this dataset, three challenging tasks and corresponding benchmarks are proposed:

  1. ECG Signal Quality Assessment: A multi-stage quality assessment model is developed, combining refined noise addition (RNA) for pre-training on public datasets and key wave detection. This approach aims to improve model performance on limited high-quality data by simulating realistic noise conditions.
  2. Individual Biometric Identification: An end-to-end contrastive learning framework is proposed. This method leverages supervised loss functions to bring similar ECG signals from the same individual closer in the feature space while pushing signals from different individuals further apart, demonstrating superiority over traditional supervised methods.
  3. Physiological Signal Analysis in Complex Driving Environments: This involves temporal segmentation of driving scenes (city vs. highway) using six-axis motion data. Subsequently, Heart Rate Variability (HRV) features are extracted from ECG signals within these segments to analyze driver stress and fatigue.

Datasets

  • OpenDriver Dataset (Newly Introduced): This is the primary contribution, comprising 3,278 driving trips, approximately 4,600 hours of data, from 81 drivers and their vehicles. It includes both ECG and six-axis motion data collected in real-world driving.
  • MIT-BIH Arrhythmia Dataset: Used for pre-training the ECG quality assessment model.
  • QT Dataset: Used for pre-training the backbone network in the ECG quality assessment task.

Key Insights and Ideas

  • Non-intrusive ECG Collection is Feasible and Effective: Collecting ECG signals via steering wheel electrodes offers a practical and less disruptive method for continuous monitoring in real driving scenarios, overcoming a major limitation of previous studies.
  • Real-World Data is Crucial for Robustness: The OpenDriver dataset's real-world nature, with its inherent noise and variability, is essential for developing models that generalize well beyond simulated environments.
  • Contrastive Learning Excels in Biometric Identification: The paper demonstrates that contrastive learning significantly outperforms traditional classification methods for individual biometric identification from ECG signals, especially when dealing with variations in driving and physiological states.
  • Driving Environment Impacts Physiological Signals: Distinct patterns in HRV features are observed between urban and highway driving, suggesting that driving task load and environment significantly influence physiological stress levels.
  • Long-Distance Driving Reveals Fatigue Patterns: HRV features show trends that can be associated with driver fatigue during prolonged driving sessions, although further research is needed to combine these with motion data for reliable detection.

Key Results

  • OpenDriver Dataset: The largest publicly available multimodal dataset for real-world driving scenarios, with 3,278 trips and ~4,600 hours of data.
  • ECG Quality Assessment: The proposed method achieves an AUC of 93.58% and an accuracy of 84.77% on the OpenDriver dataset, outperforming Wavelet Scattering+BiLSTM and ResNet.
  • Biometric Identification: The proposed contrastive learning framework achieves an accuracy of 88.98% (Acc1) and 89.61% (Acc3) using ResNet50 as a backbone, significantly outperforming traditional methods.
  • HRV Analysis: Statistically significant differences in HRV features (LF, HF, Mean HR, RMSSD, NN20) were found between urban and highway driving conditions, indicating varying stress levels.

Limitations and Future Work

  • Geographic and Demographic Limitations: Data collection was confined to specific regions, and the dataset primarily includes professional drivers, potentially limiting generalizability to diverse global driving behaviors and populations.
  • Noise in Real-World Signals: ECG signals in real driving are susceptible to various noise sources (electromagnetic interference, vehicle vibrations), and complete elimination is not feasible.
  • Lack of Direct Emotional/Fatigue Labels: The non-interventive nature means emotional and fatigue states are not directly measured, leading to a lack of hard labels for these states.
  • ECG Data Loss: Rigorous collection conditions can lead to significant ECG data loss, potentially skewing analysis.

Future Work: * Develop methods to reconstruct P and T waves in noisy ECG signals to improve arrhythmia detection. * Explore weakly supervised and transfer learning algorithms for more precise fatigue detection, emotion recognition, and driving behavior analysis. * Study the generalizability of disease models (e.g., arrhythmias) in diverse settings. * Investigate the correlation between ECG signals, motion data, and driving metrics to understand underlying mechanisms of driver states and behavior. * Develop personalized feature sets for drivers using biometric identification to create personalized fatigue detection algorithms.

Summary updated

Abstract

In modern society, road safety relies heavily on the psychological and physiological state of drivers. Negative factors such as fatigue, drowsiness, and stress can impair drivers' reaction time and decision making abilities, leading to an increased incidence of traffic accidents. Among the numerous studies for impaired driving detection, wearable physiological measurement is a real-time approach to monitoring a driver's state. However, currently, there are few driver physiological datasets in open road scenarios and the existing datasets suffer from issues such as poor signal quality, small sample sizes, and short data collection periods. Therefore, in this paper, a large-scale multimodal driving dataset for driver impairment detection and biometric data recognition is designed and described. The dataset contains two modalities of driving signals: six-axis inertial signals and electrocardiogram (ECG) signals, which were recorded while over one hundred drivers were following the same route through open roads during several months. Both the ECG signal sensor and the six-axis inertial signal sensor are installed on a specially designed steering wheel cover, allowing for data collection without disturbing the driver. Additionally, electrodermal activity (EDA) signals were also recorded during the driving process and will be integrated into the presented dataset soon. Future work can build upon this dataset to advance the field of driver impairment detection. New methods can be explored for integrating other types of biometric signals, such as eye tracking, to further enhance the understanding of driver states. The insights gained from this dataset can also inform the development of new driver assistance systems, promoting safer driving practices and reducing the risk of traffic accidents. The OpenDriver dataset will be publicly available soon.
BibTeX
@misc{liu2023opendriver,
  author = {Delong Liu and Shichao Li},
  title = {{OpenDriver: an open-road driver state detection dataset}},
  year = {2023},
  doi = {10.2139/ssrn.5013855},
  howpublished = {SSRN},
}
Figure