Abstract
Identification of high-risk driving situations is generally approached through collision risk estimation or accident pattern recognition. In this work, we approach the problem from the perspective of subjective risk. We operationalize subjective risk assessment by predicting driver behavior changes and identifying the cause of changes. To this end, we introduce a new task called driver-centric risk object identification (DROID), which uses egocentric video to identify object(s) influencing a driver's behavior, given only the driver's response as the supervision signal. We formulate the task as a cause-effect problem and present a novel two-stage DROID framework, taking inspiration from models of situation awareness and causal inference. A subset of data constructed from the Honda Research Institute Driving Dataset (HDD) is used to evaluate DROID. We demonstrate state-of-the-art DROID performance, even compared with strong baseline models using this dataset. Additionally, we conduct extensive ablative studies to justify our design choices. Moreover, we demonstrate the applicability of DROID for risk assessment.
Introduction
Traditional Advanced Driver Assistance Systems (ADAS) typically evaluate risk using objective metrics, such as the distance to the nearest vehicle or the statistical probability of a collision. However, human driving is fundamentally guided by subjective risk: a driver’s internal perception of danger that triggers specific behavioral responses, such as braking or swerving.
The primary challenge in modeling subjective risk is identifying why a driver reacted. While existing methods often rely on expensive human-labeled "importance" scores or eye-tracking data, this research introduces the Driver-centric Risk Object IDentification (DROID) task. DROID aims to identify the "cause" of a driver's behavior change using only the "effect"—the driver's observed response—as a supervision signal.
Methodology: Situation Awareness Framework
The proposed framework is structured around the psychological model of Situation Awareness (SA), which decomposes the driving task into three distinct stages: Perception, Comprehension, and Projection.
- Perception: The system processes input video to detect "Things" (dynamic objects like cars and pedestrians) and "Stuff" (static elements like lanes and road boundaries), while simultaneously estimating 3D depth to establish spatial context.
- Comprehension: Using Graph Convolutional Networks (GCNs), the model represents the environment as an "Ego-Thing" and "Ego-Stuff" graph. This allows the system to model complex spatial-temporal interactions between the ego-vehicle and surrounding elements.
- Projection: A Temporal Recurrent Network (TRN) integrates these interaction representations with the driver's high-level intention (e.g., "turning left") to forecast the driver's response (e.g., "Stop" or "Go").
Causal Reasoning and Intervention
The core innovation of DROID is its use of causal intervention to identify risk objects. Rather than relying on simple correlations, the system systematically "removes" individual objects from the scene to observe the effect on the predicted driver response.
By using partial convolutions to inpaint the background where an object was located, the model can simulate a counterfactual scenario: "What would the driver have done if this pedestrian were not there?" If the model's prediction changes from "Stop" to "Go" after an object is removed, that object is definitively identified as the functional cause of the behavior—the risk object.
Experimental Setup and Results
The researchers utilized a specialized subset of the Honda Research Institute Driving Dataset (HDD), termed HDDS, which includes layers for both goal-oriented intent and stimulus-driven responses.
Key Performance Metrics
- Response Prediction: The integrated model (GCN + TRN + Causal Augmentation) achieved a 95.44% mAP in predicting driver behavior, significantly outperforming baseline I3D+LSTM models (71.07%).
- Risk Identification: In the DROID task, the causal approach proved far more accurate than traditional attention-based models. In "Crossing Vehicle" scenarios, DROID achieved a mean accuracy (mAcc) of 32.5%, nearly doubling the performance of models relying on gaze-prediction proxies (16.8%).
Discussion and Future Work
A key insight from this work is the importance of intent-awareness. A pedestrian's risk level is highly dependent on the driver's intended path; DROID successfully incorporates this by conditioning predictions on the driver's goal. Furthermore, the use of causal intervention for data augmentation nearly halved the model's prediction error by forcing it to ignore non-essential background objects.
Current limitations include computational overhead, as the "remove-and-retest" cycle currently takes approximately 0.15s per object. Future research will focus on "single-shot" causal inference to enable real-time application and expanding the framework to handle a broader spectrum of driving maneuvers beyond binary "Stop/Go" decisions.
Summary updated