TL;DR
This paper introduces DashCop, an end-to-end system that automatically generates e-tickets for two-wheeler traffic violations using dashcam videos. It tackles the challenge of processing complex, dynamic traffic scenes by introducing novel modules for rider-motorcycle association and tracking, and a new dataset for training and evaluation.
What Problem Does This Paper Address?
Motorized two-wheelers are a dominant mode of transportation, especially in Asia-Pacific regions. However, they are also associated with a high number of accidents due to hazardous driving practices like triple riding and not wearing helmets. While automated enforcement systems (e-tickets) are crucial for improving safety, existing solutions primarily rely on static cameras, which have limitations in coverage and deterrence. Furthermore, processing dashcam footage for violation detection is challenging due to relative motion, occlusions, and the small size of objects of interest. This paper aims to bridge this gap by developing an automated system that leverages dashcam videos for violation detection and e-ticket generation.
Methods
DashCop is an end-to-end system with several key components:
- Segmentation and Cross-Association (SAC) Module: This novel module extends the YOLO-v8 framework to not only detect riders and motorcycles but also to associate them. It predicts segmentation masks for individual riders and motorcycles and then uses a "cross-object segmentation" mechanism to learn the spatial relationship between them, forming "Rider-Motorcycle (R-M) instances."
- Association-based Tracker (Cross-AssociationSORT): Building on the SORT tracker, this module tracks R-M instances across video frames. It uniquely tracks the associated rider-motorcycle pairs as a single entity, improving tracking accuracy and maintaining robust associations over time. This is achieved by using R-M association scores derived from the SAC module.
- Violation Detection Module: This module takes the tracked R-M instances and performs two key tasks:
- Helmet Detection: A helmet detection module classifies riders as wearing a 'helmet' or 'no-helmet'.
- Triple Rider Classification: A classifier determines if an R-M instance involves a 'single', 'double', or 'triple' rider.
- License Plate Recognition (LPR) and E-ticket Generation: For tracks with detected violations, an ANPR module detects and recognizes license plates. This information, along with the violation details and supporting video frames, is used to generate an e-ticket.
Datasets
- RideSafe-400 Dataset: The authors introduce a new dataset comprising 400 dashcam videos specifically curated for identifying triple riding and helmet rule violations. The dataset includes extensive annotations at the track level, covering six object classes: rider-motorcycle instance, rider, motorcycle, helmet, no-helmet, and license plate.
Key Insights and Ideas
- Rider-Motorcycle Association is Key: The paper emphasizes that effectively associating riders with their motorcycles is fundamental for accurate violation detection and tracking in complex traffic scenarios. Simply detecting riders and motorcycles independently can lead to degraded performance.
- Dashcam Videos Offer Unpredictability: Unlike static cameras, dashcams provide an element of unpredictability that can be more effective for deterrence, making them a valuable source for traffic enforcement.
- Joint Tracking Improves Robustness: Tracking R-M instances as a unified entity, rather than individual components, leads to more robust tracking performance and better association maintenance over time.
- End-to-End System for Practical Enforcement: The development of a complete system from video input to e-ticket generation addresses a significant gap in current research, moving beyond just violation detection.
Key Results
- System-Level Performance: The automated system achieves an F1-score of 72.18% for e-ticket generation. This score improves to 82.05% when a human-in-the-loop review is incorporated, highlighting the system's reliability.
- Helmet Violation Detection: The YOLOv8-x-based frame-level approach achieved the best F1-score (65.63%) in comparative evaluations, demonstrating its effectiveness.
- Triple Riding Violation: The R-M instance-based classification approach yielded the best performance, outperforming methods relying solely on counting riders.
- Rider-Motorcycle Association: The proposed SAC module achieved an association score of 84.04%, outperforming existing methods.
- Tracking Performance: The Cross-AssociationSORT tracker achieved the highest HOTA (60.41), MOTA (51.96), and IDF1 (64.58) scores, indicating superior performance in tracking R-M instances.
- License Plate Recognition: The CRNN model, trained on a combination of synthetic and real data, achieved the best performance with an accuracy of 85.71%.
Limitations and Future Work
- Limitations: The paper notes that the system struggles with detecting triple riders in scenarios with significant glare or motion blur. Helmet detection can also be affected by occlusions. The triple riding classifier sometimes misclassifies distant occluded R-M instances, leading to false positives. License plate recognition accuracy can drop for vehicles approaching from the opposite lane.
- Future Work: Not explicitly detailed, but the authors suggest the system's potential for enhancing traffic safety through automated enforcement and deterrence. The introduction of the RideSafe-400 dataset also opens avenues for future research in this domain.
Summary updated