Abstract
Reliable traffic incident recognition is a cornerstone of safe autonomous driving and Advanced Driver Assistance Systems (ADAS). However, many existing datasets lack the nuanced environmental context necessary for deep learning models to understand the causal factors behind accidents. This paper discusses the development and evaluation of the Various-perspective Traffic Incident Database (V-TIDB), a comprehensive dataset of over 9,000 videos annotated with ten layers of environmental information. By employing a multi-task learning framework with 3D-ResNet architectures, the research demonstrates that incorporating context—such as weather, road type, and incident predictability—significantly enhances model performance in recognizing and classifying traffic incidents.
1. Introduction
The evolution of autonomous vehicles (AVs) depends heavily on the ability of deep learning models to predict and respond to traffic accidents. While current systems have made strides in binary classification (accident vs. no-accident), they often struggle with the complexity of real-world scenarios. An accident is rarely an isolated event; it is the culmination of environmental variables, driver reactions, and road conditions. Most existing benchmarks provide "thin" labels that fail to capture this interplay. To address this gap, the V-TIDB was developed to provide a "thick" description of traffic scenes, enabling models to learn the "why" behind an incident rather than just the "what."
2. The V-TIDB Dataset
The V-TIDB consists of 9,062 dashcam videos (4,088 accidents and 4,974 non-accidents) sourced and preprocessed from YouTube. Unlike previous benchmarks such as DADA-2000 or DoTA, V-TIDB introduces 11 parent categories of labels, providing a multi-perspective view of each scene:
- Incident Target: Identifies the primary entities involved (e.g., cars, pedestrians, bicycles).
- Contact Level: Categorizes severity (e.g., wreck, touch, near-miss).
- Environment & Weather: Details the setting (e.g., highway, parking lot) and atmospheric conditions (e.g., rain, fog).
- Subjective Context: Includes labels like "Predictability" (whether a human could foresee the crash 2-3 seconds prior) and "Reaction."
3. Methodology
The researchers utilized 3D-ResNet (18 and 50) architectures, which are standard for video-based action recognition. The study compared two primary training approaches:
- Baseline Model: A standard binary classifier trained solely to detect the presence of an accident.
- Multi-task Learning (MTL) Model: An augmented network featuring an additional linear layer designed to predict environmental factors simultaneously with the primary accident detection task.
By forcing the model to optimize for environmental labels, the researchers aimed to guide its internal feature representation toward more relevant visual cues, such as road slickness or sudden lane changes.
4. Key Results and Insights
The experimental results confirmed that environmental context acts as a "force multiplier" for model accuracy.
4.1 Performance Gains
For the ResNet50 model, the inclusion of multi-perspective labels improved the F1-score across 9 out of 10 categories. The most significant improvement came from the "Predictability" label, which increased the F1-score from a baseline of 0.912 to 0.968. This suggests that human-centric subjective signals can serve as highly effective "teacher signals" for AI training.
4.2 Severity and Causal Factors
The model's ability to distinguish between "wrecks" (severe) and "touches" (minor) improved by 0.09 points when the "Derivation Object" (the camera's perspective) was included. Furthermore, the model achieved near-perfect accuracy in specific high-context scenarios, such as "foggy" weather or "ignored traffic light" states, indicating that the MTL approach helps the AI isolate specific causal factors.
4.3 The Near-Miss Paradox
An unexpected finding was that adding environmental information actually decreased performance for "near-miss" recognition. The authors hypothesize that near-misses are often the result of sudden driver inattention or erratic behavior that may not be strongly correlated with the broader environmental context. In these cases, the model may become "distracted" by looking for environmental clues that are irrelevant to the specific incident.
5. Limitations and Future Directions
Despite its scale, V-TIDB faces challenges common to real-world data:
- Class Imbalance: Approximately 77% of incident targets are cars, making the model less robust for rare but critical incidents involving pedestrians or cyclists.
- Subjectivity: Labels like "Predictability" and "Reaction" are assigned by human annotators and are inherently subjective, potentially embedding human biases into the model's logic.
Future research will focus on more sophisticated methods for combining multi-labels beyond simple MTL and developing strategies to better isolate driver-specific factors to improve near-miss detection.
6. Conclusion
The V-TIDB dataset demonstrates that context is essential for safer traffic AI. By moving beyond binary labels and embracing the multi-perspective complexity of the driving environment, researchers can develop models that not only detect accidents but understand the conditions that lead to them, ultimately paving the way for more reliable autonomous systems.
Summary updated