Abstract
Effective, reliable, and efficient evaluation of autonomous driving safety is essential to demonstrate its trustworthiness. Criticality metrics provide an objective means of assessing safety. However, as existing metrics primarily target longitudinal conflicts, accurately quantifying the risks of lateral conflicts—prevalent in urban settings—remains challenging. This paper proposes the Modified-Emergency Index (MEI), a metric designed to quantify evasive effort in lateral conflicts. Compared to the original Emergency Index (EI), MEI refines the estimation of the time available for evasive maneuvers, enabling more precise risk quantification. We validate MEI on a public lateral conflict dataset based on Argoverse-2, from which we extract over 1,500 high-quality AV conflict cases, including more than 500 critical events. MEI is then compared with the well-established ACT and the widely used PET metrics. Results show that MEI consistently outperforms them in accurately quantifying criticality and capturing risk evolution.
Introduction
Evaluating the safety of autonomous vehicles (AVs) is a cornerstone of their deployment. While longitudinal safety—maintaining distance on highways—is well-understood, urban environments present complex lateral conflicts, such as left turns at intersections and merging into tight gaps. Traditional metrics often struggle to capture the nuances of these 2D interactions. This paper introduces the Modified-Emergency Index (MEI), a criticality metric specifically designed to quantify the "evasive effort" required to avoid collisions in lateral conflict scenarios.
Limitations of Existing Metrics
Most current safety metrics have significant blind spots when applied to lateral urban conflicts:
- Post-Encroachment Time (PET): Measures the time gap between one vehicle exiting a conflict zone and another entering it. Being retrospective, it fails to monitor risk in real-time during the interaction.
- Anticipated Collision Time (ACT): Estimates time-to-collision based on the closest points between vehicles. However, the closest points are often not the points that will actually collide, leading to inaccurate risk assessments.
- Emergency Index (EI): An earlier iteration that used vehicle centroids for timing, lacking the geometric precision required for high-stakes near-misses.
The MEI Framework
The MEI is defined as the ratio of the spatial overlap required to be cleared to the time available to clear it: $MEI = InDepth / TEM$.
1. Interaction Depth (InDepth)
InDepth calculates the potential spatial overlap between two vehicle bounding boxes if no evasive action is taken. It accounts for vehicle geometry and can incorporate a safety buffer ($D_{safe}$).
2. Time for Evasive Maneuver (TEM)
The "Modified" aspect of MEI lies in the TEM calculation. Instead of using simple centroids, it employs a 2D-TTC (Time-to-Collision) algorithm. This identifies the specific point pair on the two vehicles predicted to collide under a constant velocity model, providing a more accurate estimation of the time remaining than the "closest point" approach used in metrics like ACT.
Validation and Key Results
The researchers validated MEI using a Lateral Conflict Resolution Dataset derived from Argoverse-2, containing over 1,500 high-quality interaction cases, including 501 critical conflicts.
- Capturing Risk Evolution: MEI accurately tracks the "rise and fall" of risk. Unlike ACT, which can be overly conservative, MEI's curve synchronizes with actual driver behavior, such as accelerating for a gap and then braking to yield.
- Superior Sensitivity: In scenarios where PET might suggest safety due to a late entry into the conflict zone, MEI correctly identifies high-risk moments during the maneuver.
- Benchmarking: The study establishes statistical thresholds for risk levels, identifying that a "Top 1%" critical event corresponds to an MEI > 2.13 m/s.
Conclusion
MEI offers a more precise, real-time picture of risk in complex urban intersections by focusing on exact predicted impact points rather than vehicle centers or retrospective time gaps. Future work will focus on correlating MEI with actual crash data and expanding its application to high-speed highway scenarios.
The open-source implementation is available at https://github.com/AutoChengh/MEI.
Summary updated