Abstract
Driver behavior is a critical factor in the vast majority of vehicle crashes. While naturalistic driving studies provide invaluable data for safety research, the sensitive biometric information contained in video footage often restricts data access to secure enclaves. This paper presents an overview of the "Video Data Privacy Task" at MediaEval 2021. The task establishes a benchmarking framework to evaluate video masking algorithms that aim to de-identify drivers while preserving essential behavioral cues, such as facial expressions and gaze patterns. By formalizing the privacy-utility tradeoff, this work seeks to enable broader access to critical transportation safety datasets.
Introduction
Approximately 90% of vehicle crashes are attributed to driver behavior. To understand the root causes of these incidents, researchers rely on "naturalistic driving" data—extensive video recordings captured from within the vehicle cabin. However, these datasets, such as the Second Strategic Highway Research Program (SHRP2), contain highly sensitive personal information. Consequently, they are often locked in "secure enclaves," severely limiting the research community's ability to analyze them.
The fundamental challenge lies in the privacy-utility tradeoff: obscuring a driver’s identity to protect their privacy without compromising the subtle facial cues and gaze patterns necessary to determine why a crash occurred. The Video Data Privacy Task at MediaEval 2021 was designed to address this by inviting participants to develop and test sophisticated de-identification methods.
Methodology and Datasets
The task utilized a specialized dataset prepared by Oak Ridge National Laboratory (ORNL), featuring both high- and low-resolution video captured with hardware identical to that used in the SHRP2 study. Unlike raw safety data, these sessions involved choreographed behaviors—such as eating, singing, or waving—providing a "ground truth" for action preservation testing.
The evaluation framework consists of two primary tracks:
1. De-identification Testing
This track measures how effectively an algorithm hides the driver's identity.
- Human Evaluation: Volunteers attempt to match a masked video to one of five high-resolution static photos of the "internal face" (cropped to exclude peripheral cues like hair).
- Automated Evaluation: Standard face recognition and detection algorithms are employed to determine if automated systems can still identify the driver. Success is quantified using the Intersection over Union (IoU) of detected faces.
2. Action Preservation Testing
This track assesses the utility of the masked video for behavioral research.
- Human Evaluation: Participants identify the "most obvious" action performed by the driver in the masked footage.
- Automated Evaluation: A deep-learning gaze estimator extracts gaze vectors from both original and masked videos. The Euclidean distance between these vectors serves as a metric; a lower distance indicates that the masking process preserved the driver's visual attention patterns.
Key Insights and Threat Modeling
A critical component of the task is the explicit definition of the "threat model." The authors assume an attacker with basic computational resources who does not know the driver personally. This realistic grounding ensures that privacy requirements are practical rather than theoretical.
Furthermore, the focus on gaze as a proxy for utility is a significant contribution. In driving safety research, knowing where a driver is looking is often more vital than seeing the specific movement of their mouth or eyebrows. By prioritizing gaze vector preservation, the task aligns with the actual needs of safety analysts.
Conclusion and Future Directions
As an overview of a competition task, this paper establishes a robust baseline for measuring success in driver de-identification. Success is defined by the delta in identification accuracy and action categorization between unmasked and masked footage.
While the use of choreographed data and the challenges of demographic matching in evaluation remain limitations, the framework provides a clear path forward. Successful methodologies emerging from this task have the potential to "unlock" massive, restricted datasets like SHRP2, allowing the global research community to collaborate on improving road safety without compromising individual privacy.
Summary updated