Robustness Analysis of Pedestrian Detectors for Surveillance

Yuming Fang, Guanqun Ding, Yuan Yuan, Weisi Lin, Haiwen Liu

Abstract

While pedestrian detection has seen significant progress, performance often degrades sharply in real-world surveillance scenarios characterized by low-quality video. This paper presents a systematic study of detector robustness using the newly developed Distorted Surveillance Video Data Set (DSurVD). By introducing a novel stability metric and the "Robustness Quadrangle" visualization, the authors reveal critical performance thresholds and identify feature sets that enhance resilience against common distortions like compression, noise, and lighting variations.

Introduction

In laboratory settings, pedestrian detectors are typically evaluated on high-resolution, well-lit datasets. However, real-world surveillance systems frequently operate under sub-optimal conditions. Video footage is often heavily compressed to conserve bandwidth, plagued by sensor noise, dimmed by poor lighting, or downsampled to low resolutions. While much research has focused on handling content challenges—such as occlusions or varied poses—there has been a lack of systematic methodology for measuring robustness to video quality degradation.

This research addresses this gap by investigating how much a video can be degraded before a detector fails and whether that failure is graceful or catastrophic.

The DSurVD Dataset

To facilitate this study, the authors created the Distorted Surveillance Video Data Set (DSurVD). This dataset builds upon five high-quality reference sequences from established benchmarks like PETS09, TownCentre, and ParkingLot. From these, 52 distorted versions were generated for each sequence, covering four primary categories: H.264 compression, resolution reduction, white noise, and brightness variation.

Figure 1: Examples of the various distortions included in the DSurVD dataset, ranging from low brightness to heavy compression and low resolution.

The resulting 265 video sequences, complete with manually labeled ground truth, provide a rigorous testing ground for evaluating the limits of modern detection algorithms.

Methodology: Quantifying Robustness

The paper introduces two key innovations for assessing detector performance:

  1. Stability Metric ($S$): Rather than focusing solely on baseline accuracy, this metric tracks the rate of performance change as video quality decreases. A robust detector should exhibit a slow, monotonic decline in accuracy, avoiding erratic oscillations in performance.
  2. Robustness Quadrangle: This visualization tool maps a detector's stability across four dimensions: Compression, Resolution, Noise, and Brightness. The area and shape of the resulting polygon provide an immediate visual summary of a detector's strengths and weaknesses relative to its baseline performance.
Figure 3: Quantitative characterization of the DSurVD distortions, showing how metrics like PSNR and compression ratio evolve across different quality levels.

Key Insights and Results

The evaluation of seven classic detectors (including HOG, LatSVM, and ACF) yielded several critical insights:

  • The "Critical Quality Point": Most detectors maintain stable performance until a specific threshold of degradation is reached, after which accuracy "falls off a cliff." Extending this threshold is essential for reliable real-world applications.
  • Feature-Level Resilience: Adding Local Binary Pattern (LBP) features to HOG significantly improved stability against H.264 compression artifacts. Similarly, PCA-HOG (used in LatSVM) proved effective at filtering out white noise by focusing on principal image structures.
  • Resolution vs. Model Height: Detectors trained on smaller model heights (e.g., 50 pixels) demonstrated superior stability as resolution dropped, though this often comes at the cost of peak accuracy on high-resolution footage.
  • The Lighting Challenge: While HOG-based features are naturally robust to varying brightness due to their reliance on gradients, over-exposed (extremely bright) video remains a major failure point for nearly all tested detectors.

Conclusion

This work highlights that high baseline accuracy does not guarantee real-world reliability. The DSurVD benchmark and the Robustness Quadrangle provide necessary tools for developing the next generation of surveillance-ready detectors. Future research should focus on bridging the gap between high-resolution accuracy and low-resolution stability, as well as addressing the persistent challenges posed by over-exposed environments. Applying these robustness metrics to modern deep learning architectures remains a vital next step for the field.

Summary updated

Abstract

To obtain effective pedestrian detection results in surveillance video, there have been many methods proposed to handle the problems from severe occlusion, pose variation, clutter background, \emph{etc}. Besides detection accuracy, a robust surveillance video system should be stable to video quality degradation by network transmission, environment variation, etc. In this study, we conduct the research on the robustness of pedestrian detection algorithms to video quality degradation. The main contribution of this work includes the following three aspects. First, a large-scale Distorted Surveillance Video Data Set (DSurVD) is constructed from high-quality video sequences and their corresponding distorted versions. Second, we design a method to evaluate detection stability and a robustness measure called Robustness Quadrangle, which can be adopted to visualize detection accuracy of pedestrian detection algorithms on high-quality video sequences and stability with video quality degradation. Third, the robustness of seven existing pedestrian detection algorithms is evaluated by the built DSurVD. Experimental results show that the robustness can be further improved for existing pedestrian detection algorithms. Additionally, we provide much in-depth discussion on how different distortion types influence the performance of pedestrian detection algorithms, which is important to design effective pedestrian detection algorithms for surveillance. The DSurVD data set can be download from BaiduYunDisk, https://pan.baidu.com/s/1I9Kqj8rmubOYu7bkBfkUpA, Password: lqmc
BibTeX
@article{fang2018robustness,
  author = {Yuming Fang and Guanqun Ding and Yuan Yuan and Weisi Lin and Haiwen Liu},
  title = {{Robustness Analysis of Pedestrian Detectors for Surveillance}},
  journal = {IEEE Access},
  year = {2018},
  doi = {10.1109/access.2018.2840329},
}
Figure