High-frequency crowd insights for public safety and congestion control

Karthik Nandakumar, Sebastien Blandin, Laura WynterarXiv 2019 · OpenReview

TL;DR

This paper presents a system that leverages existing CCTV infrastructure to provide real-time insights into crowd behavior, including density estimation and incident detection. By fusing multiple computer vision techniques, the system aims to enhance public safety and optimize urban infrastructure management in increasingly crowded environments.

What Problem Does This Paper Address?

As urbanization accelerates, cities worldwide face mounting pressure on public infrastructure, leading to increased crowd density in public spaces. This surge in crowds can degrade efficiency and compromise public safety, especially during large events or at critical transport hubs. Effectively managing these crowds requires precise, real-time understanding of their density, flow, and potential for incidents. Traditional methods like GPS tracking are insufficient for dense, dynamic environments, and manual monitoring is impractical. This paper tackles the challenge of extracting actionable crowd insights from ubiquitous CCTV footage, bridging the gap between raw video data and effective urban management.

Methods

The proposed system comprises two main modules: crowd flow and density estimation, and crowd anomaly detection.

  • Crowd Flow and Density Estimation: This module employs an "adaptive fusion" approach.
  • For low crowd densities, it uses a people detection method that identifies individual pedestrians based on full-body, head-and-shoulder, and head features. This is extended with tracking to estimate pedestrian flow, categorizing it into incoming and outgoing.
  • For high crowd densities, where occlusion is a major issue, it uses regression-based counting. This method extracts image features (like gradients, texture, perspective-normalized foreground/edge ratios) and uses a linear regression model (or deep learning features from AlexNet for more computational power) to directly estimate crowd density.
  • The adaptive fusion intelligently combines these two approaches. It prioritizes detection-based counts in sparse scenes and regression-based counts in dense scenes, automatically assigning importance based on where pedestrian detections are feasible. This fusion aims to improve accuracy across varying crowd conditions.
  • Crowd Incident Detection: This module focuses on identifying abnormal crowd behavior, characterized by rapid, directional movement.
  • Aggregate flow patterns: Optical flow is used to capture coarse crowd movement. Histograms of flow fields within image blocks are computed and concatenated into a global feature vector, which is fed into a Support Vector Machine (SVM) classifier to detect events like fights.
  • Features from selected interest points: Specific interest points are tracked for short intervals to generate "tracklets." Spatio-temporal features are then extracted from these tracklets, focusing on patterns like collectiveness (individuals moving together) and conflict (individuals moving in different directions). An SVM classifier uses these features to detect violence/fights.
  • The confidence scores from both incident detection methods are combined to produce an overall confidence measure for abnormal crowd events.

Datasets

  • Crowd Flow and Density Estimation: The system was trained on 90% of data and tested on 10%. The paper mentions using data from the ShangaiTech dataset for illustrative figures. Specific details on the training/testing split and the nature of the data (e.g., urban train stations, indoor/outdoor locations) are provided in the "Results" section.
  • Crowd Incident Detection: The primary dataset used for evaluation is the Violent Flows dataset, which contains 246 YouTube videos. The authors also collected a small, independent test set of 10 YouTube videos to further evaluate their system.

Key Insights and Ideas

  • Ubiquitous CCTV as a Sensing Backbone: The paper highlights the immense potential of leveraging existing CCTV networks, which are already deployed globally, as a cost-effective and scalable sensing mechanism for urban monitoring.
  • Adaptive Fusion for Robustness: The core innovation in density estimation is the adaptive fusion of detection-based and regression-based methods. This intelligent combination allows the system to maintain accuracy across a wide spectrum of crowd densities, overcoming the limitations of each individual method.
  • Beyond Simple Counting: Behavioral Analysis: The system moves beyond just counting people to understanding their behavior. By analyzing flow patterns and tracklet dynamics, it can detect specific types of incidents, offering a more nuanced understanding of crowd situations.
  • Real-world Applicability: The paper emphasizes that the methods are designed for practical urban monitoring, aiming for high accuracy in real-world scenarios rather than just controlled laboratory settings.

Key Results

  • Crowd Flow and Density Estimation: The adaptive fusion model significantly improves upon base models, especially in high-crowd scenarios. While errors increase with crowd level, the fusion model provides results within 5 persons of ground truth for low crowds and within 25% for dense crowds, outperforming off-the-shelf software by 10-20%.
  • Crowd Incident Detection: The system achieved an Area Under the Receiver Operating Characteristic Curve (AUC) of 87.9% on the Violent Flows dataset, an improvement over the 85% reported in prior work [4]. On their custom YouTube test set, the method detected fights with a confidence value greater than 0.4 in all 7 fight samples. The average detection time lag for violence was found to be 10 seconds.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that accuracy in crowd density estimation decreases with increasing crowd levels.
  • The incident detection module's performance is evaluated on datasets that may not cover all possible types of abnormal crowd behavior.
  • Human annotation reliability was found to be an issue below a certain pixel threshold for individual features.
  • Future Work:
    • The authors suggest that computing power increases will further improve accuracy.
  • The work demonstrates the feasibility of estimating crowd level with high accuracy in practical urban monitoring contexts, implying potential for broader deployment and further refinement.
  • The fusion of crowd density and incident detection suggests a pathway towards more comprehensive real-time urban safety systems.

Summary updated

Abstract

We present results from several projects aimed at enabling the real-time understanding of crowds and their behaviour in the built environment. We make use of CCTV video cameras that are ubiquitous throughout the developed and developing world and as such are able to play the role of a reliable sensing mechanism. We outline the novel methods developed for our crowd insights engine, and illustrate examples of its use in different contexts in the urban landscape. Applications of the technology range from maintaining security in public spaces to quantifying the adequacy of public transport level of service.
BibTeX
@misc{nandakumar2019high,
  author = {Karthik Nandakumar and Sebastien Blandin and Laura Wynter},
  title = {{High-frequency crowd insights for public safety and congestion control}},
  year = {2019},
  eprint = {1904.10180},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1904.10180},
}
Figure