TL;DR
This paper introduces RaDA, a novel dataset for in-cabin driving activity recognition using UWB radar, and proposes a deep learning approach (ResNet-18 and LSTM) that achieves promising generalization to unseen drivers. The work highlights the potential of radar as a privacy-preserving alternative to vision systems for monitoring driver states like distraction.
What Problem Does This Paper Address?
As vehicles become more automated, ensuring driver engagement and preventing distraction is paramount for safety. Current in-cabin monitoring systems often rely on cameras or eye-tracking, which can raise privacy concerns or fail to fully capture a driver's attentiveness. While radar offers a privacy-friendly and robust solution, research into its application for nuanced driving activity recognition, especially for generalizing to new users, is limited. Existing radar-based HAR datasets often use random data splitting, which can lead to models overfitting to specific individuals rather than learning generalizable patterns. This paper aims to address this gap by developing a radar-based system that can recognize various driving activities, including distracted states, and crucially, generalize to drivers not seen during training.
Methods
The core of the proposed approach involves using Ultra-Wideband (UWB) radar to capture Doppler signals, which are then processed into range-Doppler maps. These maps serve as input to deep neural networks. Specifically, the authors employ a ResNet-18 architecture to extract features from these maps and an LSTM (Long Short-Term Memory) network to process the temporal sequences of these features.
A key methodological contribution is the use of Interquartile Range (IQR) normalization. This technique is applied to the range-Doppler bins to handle outliers (e.g., strong reflections from car seats) by replacing values exceeding a certain threshold with the maximum value within the acceptable range. The IQR parameters are calculated only on the training data and then applied to both training and validation sets, ensuring a consistent normalization strategy that adapts to the data's scale without requiring radar-specific pre-processing. This is crucial for real-time application and transferability across different radar systems.
For evaluation, the paper emphasizes the importance of leave-one-participant-out cross-validation. This rigorous approach ensures that the model's ability to generalize to entirely new individuals is tested, moving beyond the limitations of random splitting which can lead to data leakage and inflated performance metrics.
Datasets
The paper introduces the RaDA (Radar Driving Activity) dataset. This dataset was collected using a UWB radar sensor (Xethru X4M02) in a driving simulator. It comprises approximately 60 minutes of data from ten participants performing six distinct driving activities:
- Driving
- Autopilot
- Sleeping
- Driving & smartphone utilization
- Smartphone utilization
- Talking to passenger
The dataset is made publicly available to encourage further research and benchmarking in the field.
Key Insights and Ideas
- Radar's Privacy Advantage: The paper strongly advocates for UWB radar as a privacy-preserving alternative to vision-based systems for in-cabin monitoring, a critical factor for user acceptance.
- Generalization is Key: The study highlights that random data splitting, common in many HAR studies, leads to models overfitting to individual characteristics. Rigorous cross-validation (leave-one-participant-out) is essential for evaluating true generalization capabilities.
- IQR Normalization Boosts Performance: The IQR normalization technique significantly improves the generalization performance of both ResNet-18 and LSTM models, particularly for unseen drivers. This simple yet effective method helps in handling radar data's inherent variability.
- Distracted Driving Detection: The research demonstrates that radar can effectively distinguish between normal driving and various forms of distracted driving (e.g., smartphone use, sleeping), which is vital for safety applications.
Key Results
- ResNet-18 with IQR Normalization: Achieved the highest average classification accuracy of 71.3% using leave-one-out cross-validation.
- LSTM with IQR Normalization: Achieved an average classification accuracy of 67.2% using leave-one-out cross-validation.
- Random Splitting Overfitting: Both ResNet-18 and LSTM achieved near-perfect accuracy (100%) with random splitting, underscoring the overfitting issue.
- Baseline Comparison: A re-implementation of a previous radar-based method ([32]) performed poorly on the RaDA dataset, achieving only around 26.3% accuracy with random splitting and 36.92% with cross-validation, suggesting the RaDA dataset presents a more challenging benchmark for generalization.
- Real-time Capability: The inference time for ResNet-18 was 10.38 ms, and for LSTM was 55.5 ms, indicating the feasibility of real-time application.
Limitations and Future Work
- Generalization Challenge: The paper explicitly states that radar-based activity recognition cannot be easily generalized to new, unseen drivers, as evidenced by the significant drop in performance when moving from random splitting to cross-validation.
- Confusion Between Similar Activities: High confusion was observed between classes that share similar underlying driving behaviors (e.g., "driving," "driving & smartphone utilization," and "talking to passenger"). This ambiguity needs to be addressed for robust safety-critical responses.
- Future Work Suggestions:
- Conduct large-scale evaluations with more participants and diverse driving conditions.
- Investigate methods to reduce ambiguity between classes with high confusion levels.
- Explore advanced deep learning architectures and feature extraction techniques.
Summary updated