LATTE: A Real-time Lightweight Attention-based Traffic Accident Anticipation Engine

TL;DR

This paper introduces LATTE, a novel engine for real-time traffic accident anticipation that balances high accuracy with computational efficiency. It achieves this through a lightweight attention-based architecture and a unique alert system, making it suitable for deployment on resource-constrained autonomous driving platforms.

What Problem Does This Paper Address?

Predicting traffic accidents in real-time is a critical but challenging task for autonomous driving systems. Existing methods often struggle with high computational costs, limiting their deployment on edge devices, or fail to adequately capture the complex interactions and evolving nature of traffic scenarios. Furthermore, many systems lack effective ways to communicate potential hazards to passengers, hindering trust and situational awareness. This paper aims to bridge this gap by developing a system that is both accurate and efficient, while also providing clear, actionable alerts.

Methods

LATTE employs a multi-component architecture designed for efficiency and effectiveness:

• Efficient Multiscale Spatial Aggregation (EMSA): This module efficiently captures spatial features across different scales by partitioning input tensors and using parallel convolutional branches. This approach reduces computational cost while preserving crucial spatial details.
• Memory Attention Aggregation (MAA): To model temporal dependencies, MAA uses memory units with reduced dimensionality. This allows for efficient extraction of abstract temporal information without the quadratic scaling issues of traditional self-attention mechanisms, crucial for long video sequences.
• Auxiliary Self-Attention Aggregation (AAA): This module adaptively weights features based on contextual relevance, amplifying accident-related signals and suppressing irrelevant ones. It uses depthwise separable convolutions to maintain computational tractability and preserve fine-grained spatial semantics.
• Flamingo Alert-Assisted System (FAA): This novel component integrates a vision-language model (Flamingo) with GPT-3.5 to generate real-time, semantically transparent verbal hazard alerts. This enhances passenger situational awareness and builds trust by providing clear, human-understandable warnings.

Datasets

LATTE was evaluated on three publicly available datasets:

• CCD (Car Crash Dataset): Provides detailed annotations of environmental factors, ego-vehicle involvement, and causal mechanisms.
• DAD (Dashcam Accident Dataset): Features 720p dashcam footage from urban environments, including various accident types and edge cases.
• A3D (AnAn Accident Detection Dataset): Documents abnormal road events in urban settings, maintaining similar temporal and structural configurations to DAD.

Key Insights and Ideas

• Efficiency is Key for Deployment: The paper strongly emphasizes that high accuracy alone is insufficient for real-world autonomous driving. Computational efficiency is paramount for deployment on resource-constrained edge devices.
• Attention Can Be Lightweight: LATTE demonstrates that attention mechanisms, often associated with high computational cost, can be designed in a lightweight and efficient manner without sacrificing performance.
• Multimodal Alerts Enhance Trust: Integrating a vision-language model for verbal alerts significantly improves the practical utility of accident anticipation systems by providing clear, actionable information to occupants.
• Component Synergy: The combination of EMSA, MAA, and AAA modules is crucial for achieving both accurate spatial-temporal feature extraction and efficient processing.

Key Results

• State-of-the-Art Performance: LATTE achieves state-of-the-art performance on the DAD dataset, with an Average Precision (AP) of 89.74%.
• Improved Early Warning: It demonstrates a 5.4% improvement in mean Time-To-Accident (mTTA) compared to the second-best model on DAD.
• Significant Efficiency Gains: LATTE achieves a 93.14% reduction in FLOPs and a 31.58% decrease in parameter count compared to other advanced models, enabling real-time operation.
• Robust Generalization: The model shows superior cross-domain generalization on DAD, validating its reliability across diverse driving conditions.
• Effective Alerting: The FAA system successfully generates timely verbal alerts, as visualized in accident-positive scenarios.

Limitations and Future Work

• Data Limitations: The paper identifies a failure case where LATTE struggled to predict a motorcycle-barrier accident due to limited training data for vehicle-infrastructure interactions.
• Future Directions:
  • Expand training dataset diversity to include under-represented scenarios.
• Enhance multimodal sensing capabilities (e.g., LiDAR, radar, V2X).
• Integrate vehicular pose estimation and driver state analysis (behavioral, physiological) for more comprehensive anticipation.
• Explore edge-optimized model distillation and domain-adaptive visual processing.
• Investigate uncertainty-aware language generation for more robust alerts.

@article{li2025latte,
  author = {Zhenning Li and Jiaxun Zhang and Yanchen Guan and Chengyue Wang and Haicheng Liao and Guohui Zhang and Zhenning Li},
  title = {{LATTE: A Real-time Lightweight Attention-based Traffic Accident Anticipation Engine}},
  journal = {Information Fusion},
  year = {2025},
  doi = {10.1016/j.inffus.2025.103173},
}