TL;DR
This paper introduces a novel machine learning system that uses standard RGB cameras to estimate a driver's Blood Alcohol Concentration (BAC) by analyzing facial features. The system achieves 75% accuracy in classifying three levels of impairment (sober, low, and severe intoxication) and introduces a new, large-scale dataset of simulated driving scenarios with varying BAC levels.
What Problem Does This Paper Address?
Drunk driving remains a significant cause of road fatalities worldwide. Current methods for detecting impaired drivers, such as random breath tests, are reactive and don't prevent impaired individuals from getting behind the wheel. While some research explores driver behavior or specialized sensors for impairment detection, these approaches often require extensive vehicle modifications or are limited in their applicability. There's a clear need for a non-intrusive, cost-effective, and widely deployable solution that can identify alcohol-induced impairment early, ideally before a driver poses a risk. Existing datasets are also limited in size and scope, hindering the development of robust models.
Methods
The researchers developed a machine learning framework centered on analyzing facial features captured by standard RGB cameras. The core steps involve:
- Feature Generation: Extracting key facial attributes known to be affected by alcohol, including head movement, gaze direction, eye aspect ratio (EAR), and facial landmark variations (p-scale). These features are derived from OpenFace software and processed over time windows to capture behavioral patterns.
- Feature Pre-processing and Normalization: Calculating the eye aspect ratio (EAR) and normalizing features using a Z-score approach. This normalization accounts for individual variability in alcohol response by using each participant's sober state as a reference.
- Model Design and Training: Employing a Long Short-Term Memory (LSTM) neural network, chosen for its ability to process sequential data and capture temporal dependencies in facial features. Hyperparameters were optimized using Bayesian optimization, and the model was trained using a 5-fold cross-validation strategy.
- Classification: The trained LSTM model classifies drivers into three categories: sober (0.00 BAC), low alcohol intoxication (0.05-0.07 BAC), and severe alcohol intoxication (above 0.08 BAC).
Datasets
The paper introduces a newly compiled dataset consisting of simulated driving videos from 60 participants. This dataset includes:
- Simulated Driving Scenarios: Participants drove in a simulator under three distinct BAC levels: sober (0.00 g/dL), low intoxication (0.05-0.07 g/dL), and severe intoxication (above 0.08 g/dL).
- Multi-modal Data: While the core ML model uses standard RGB videos of the driver's face, the dataset also includes IR and 3D facial videos, rear-view RGB videos (for posture and steering), driving event logs, and screen recordings. This rich dataset is made available for future research.
This dataset is notable for its size (60 participants, over 30 hours of data) and the inclusion of multiple intoxication levels, aiming to be more representative of real-world scenarios than previous smaller datasets.
Key Insights and Ideas
- Facial Cues are Sufficient: The study demonstrates that standard RGB cameras, without specialized sensors, can capture sufficient facial cues to detect alcohol-induced impairment. This significantly lowers the barrier to adoption.
- Temporal Dynamics Matter: Analyzing features over time windows (specifically 150 frames) proved more effective than static analysis, highlighting that gradual behavioral shifts are key indicators of intoxication.
- Individual Variability is Key: The normalization strategy, using each participant's sober state as a baseline, is crucial for handling the diverse ways alcohol affects individuals.
- EAR and Head Movement are Strong Indicators: Reduced Eye Aspect Ratio (EAR) and increased head movement variability were identified as significant indicators of intoxication, aligning with existing literature on alcohol's effects on ocular and motor functions.
Key Results
- Overall Accuracy: The system achieved an overall accuracy of 75% for classifying drivers into sober, low AII, and severe AII categories.
- Class-Specific Performance: The model performed best in identifying "sober" drivers (70.94% TP rate), with slightly lower recall for "low AII" and "severe AII" states.
- Confusion: A moderate confusion (21% ± 0.5) was observed between "low AII" and "severe AII" states.
- Robustness: Cross-validation and multiple runs with different random seeds confirmed the model's consistent performance and generalizability, with a small standard deviation of 0.01 in accuracy.
- Feature Importance: "P-scale," "gaze angle," and "EAR" were found to be the most significant features, while "head pose" had minimal impact.
Limitations and Future Work
- Simulator vs. Real-World: The study acknowledges the inherent limitations of simulator-based research, though it argues that simulators effectively replicate real-world driving changes due to alcohol. Participants' potential to improve performance with simulator use was mitigated by an initial practice session.
- Normalization Challenges: The current normalization approach relies on individual sober-state data. While suitable for known drivers, it poses challenges for scenarios like rental cars. Future work could explore normalizing against an average sober state across a larger population.
- Manual Feature Engineering: The current system relies on manually designed features. The authors suggest transitioning to an end-to-end learning approach for potentially improved performance.
- Data Availability: The collected data will be available upon request for non-profit and research purposes, with data availability not before 2025.
Summary updated