Video Analytics in Smart Transportation for the AIC'18 Challenge

Ming-Ching Chang, Yi Wei, Nenghui Song, Siwei Lyu

TL;DR

This paper presents a comprehensive video analytics system for smart transportation, tackling three key challenges: traffic flow analysis, anomaly detection (stalled vehicles/crashes), and vehicle re-identification across multiple cameras. The authors demonstrate robust performance on the AIC'18 challenge, highlighting the potential of AI for safer, more efficient urban mobility.

What Problem Does This Paper Address?

Smart transportation systems are increasingly relying on video analytics to extract valuable insights from ubiquitous street cameras. However, accurately analyzing complex traffic scenarios presents significant challenges. These include precisely tracking vehicles to estimate flow and speed, reliably detecting rare but critical anomalies like accidents, and identifying individual vehicles across vast amounts of video data from multiple viewpoints. The NVIDIA AIC'18 Challenge aimed to push the boundaries in these areas, and this paper details the authors' approach to tackling these multifaceted problems.

Methods

The authors developed a system that addresses three distinct tracks of the AIC'18 challenge:

  • Track 1: Traffic Flow Analysis: This involved a pipeline for automatic traffic flow analysis. Key components include:
  • Vehicle Detection: Utilizing a Faster R-CNN inception ResNet v2 atrous model, pre-trained on COCO and fine-tuned on the UA-DETRAC dataset.
  • Camera Calibration: A one-time calibration process using landmark-based methods to estimate the camera projection matrix, crucial for projecting 2D detections to 3D ground planes.
  • Vehicle Tracking: A tracking-by-detection approach employing Kalman filters on projected trajectories, with robust association using IoU and Hungarian assignment.
  • Speed Estimation: On-line speed estimation with temporal median and Gaussian smoothing, followed by off-line refinement.
  • Track 2: Traffic Anomaly Detection: The focus here was on identifying stalled vehicles and crashes. The approach is probabilistic and rule-based, avoiding explicit vehicle detection/tracking for anomalies.
  • ROI Extraction: Traffic lanes are identified and masked using motion flow to define regions of interest (ROIs) for anomaly detection.
  • Block-based Features: The ROI is divided into blocks, and features like foreground area size and motion speed are extracted.
  • Anomaly Rules: Stalled vehicles are identified based on sustained foreground area and near-zero motion speed, with a minimum time lapse to filter out normal stops (e.g., at traffic lights).
  • Perspective Scale Handling: Scale rectification is applied to account for perspective changes.
  • Track 3: Vehicle Re-identification: This track aimed to match vehicles across multiple cameras and long durations.
  • Vehicle Detection and Tracking: Similar to Track 1, to obtain vehicle images.
  • Deep Feature Extraction: A modified ResNet-50 model, fine-tuned on the VeRi dataset using a triplet loss, is used to extract deep embedding features.
  • Pairwise Matching: A progressive matching strategy is employed, starting with pairwise matches and extending to triplets and quadruplets across all four test sites.

Datasets

  • Training:
    • UA-DETRAC dataset for vehicle detection.
  • VeRi dataset for training the triplet-based vehicle feature extractor for re-identification.
  • Evaluation:
    • NVIDIA AIC'18 Challenge dataset for all three tracks. This dataset consists of real-world traffic videos from various locations, under diverse conditions (weather, lighting, traffic density), and with varying camera viewpoints.

Key Insights and Ideas

  • Unified Framework: The paper demonstrates the feasibility of a unified framework that can address multiple critical aspects of smart transportation video analytics.
  • Robustness to Real-World Conditions: The methods are designed to handle the inherent challenges of real-world traffic videos, such as occlusions, varying lighting, and camera motion.
  • Importance of Calibration: Accurate camera calibration is highlighted as a crucial prerequisite for reliable 3D spatial analysis, including speed estimation.
  • Probabilistic Anomaly Detection: The success of a probabilistic, rule-based approach for anomaly detection, which bypasses the need for precise vehicle tracking in anomaly scenarios, is a key insight.
  • Limitations of Current Re-identification: The results for Track 3 underscore the significant challenges in vehicle re-identification over long durations and across multiple sites, especially when license plates are not consistently visible.

Key Results

  • Track 1 (Traffic Flow Analysis): Achieved a score of 0.6226, ranking 11th out of 13 teams. The system demonstrated a detection rate of 0.8519 and a speed estimation RMSE of 10.3405 MPH.
  • Track 2 (Traffic Anomaly Detection): Achieved a score of 0.4951, ranking 3rd out of 7 teams. The system achieved an F1-score of 0.6286 and a normalized RMSE of 48.3406 seconds for anomaly time.
  • Track 3 (Vehicle Re-identification): Achieved a score of 0.0074, ranking 4th out of 10 teams. The authors note that their method recalled some ground-truth vehicles but did not successfully re-acquire them across all four sites, indicating current limitations.

Limitations and Future Work

  • Track 3 Evaluation Bias: The authors suggest that the evaluation for Track 3 might be biased due to a small number of ground-truth controlled vehicles and not fully accounting for identical or similar vehicles appearing over long durations.
  • Current Re-identification Limitations: The paper explicitly states that the results for Track 3 highlight the current limitations of state-of-the-art computer vision methods for practical vehicle re-identification in real-world scenarios, with significant room for improvement.
  • Future Work: The authors plan to continue refining their algorithms on larger real-world datasets and live beta sites. They also aim to improve the multi-camera vehicle re-identification capabilities using deep progressive searches with early jump-out.

Summary updated

BibTeX
@inproceedings{chang2018video,
  author = {Ming-Ching Chang and Yi Wei and Nenghui Song and Siwei Lyu},
  title = {{Video Analytics in Smart Transportation for the AIC'18 Challenge}},
  booktitle = {Workshops @ CVPR},
  year = {2018},
  url = {https://openreview.net/forum?id=S1-QokzuZr},
}

Loading related content...

Figure