Surveillance Face Recognition Challenge

Zhiyi Cheng, Xiatian Zhu, Shaogang GongarXiv 2018 · OpenReview

Abstract

Face recognition (FR) is one of the most extensively investigated problems in computer vision. Significant progress in FR has been made due to the recent introduction of larger-scale FR challenges, particularly with constrained social media web images, e.g., high-resolution photos of celebrity faces taken by professional photo-journalists. However, the more challenging FR in unconstrained and low-resolution surveillance images remains largely under-studied. To facilitate more studies on developing FR models that are effective and robust for low-resolution surveillance facial images, we introduce a new Surveillance Face Recognition Challenge, the QMUL-SurvFace benchmark. This new benchmark is the largest and, more importantly, the only true surveillance FR benchmark to our best knowledge, where low-resolution images are not synthesized by artificial down-sampling of native high-resolution images. This challenge contains 463,507 face images of 15,573 distinct identities captured in real-world uncooperative surveillance scenes over wide space and time. We benchmark the FR performance on this challenge using five representative deep learning face recognition models. We show that the current state of the arts are still far from being satisfactory to tackle the under-investigated surveillance FR problem in practical forensic scenarios. On the new Surveillance FR Challenge, the top-performing CentreFace deep learning FR model on the MegaFace benchmark can now only achieve a 13.2% success rate (at Rank-20) at a 10% false alarm rate.

The Reality of Surveillance Face Recognition

While face recognition is often considered a "solved" problem in high-resolution celebrity datasets, there exists a massive performance gap when these models are applied to real-world surveillance. Most progress in the field has been driven by datasets like LFW or MegaFace, which consist of high-resolution, professionally shot photos. In these constrained environments, deep learning models achieve near-perfect accuracy.

However, real-world surveillance involves "native" low-resolution imagery, motion blur, extreme poses, and poor lighting. Furthermore, surveillance is inherently an open-set problem: in a crowded public space, most people are "distractors" not present in a gallery database. The research community has often been over-optimistic because it lacks a large-scale, realistic benchmark that captures these specific challenges.

Comparison between standard face recognition datasets (left) and the native surveillance images of the QMUL-SurvFace benchmark (right).

The QMUL-SurvFace Benchmark

To address this gap, we introduce QMUL-SurvFace, the largest "true" surveillance benchmark to date. Unlike previous datasets that artificially degrade high-resolution photos, QMUL-SurvFace uses native surveillance images derived from 17 real-world person re-identification datasets.

  • Scale: 463,507 images of 15,573 unique identities.
  • Diversity: The dataset captures identities across various geographic regions, with a significant concentration in China, reflecting real-world surveillance deployments.
Geographic distribution of face IDs (top) and total images (bottom) within the QMUL-SurvFace dataset.

Experimental Evaluation

We evaluated the problem through three main lenses:

  1. Benchmarking Deep FR Models: We tested five representative models—DeepID2, CentreFace, FaceNet, VggFace, and SphereFace—to see how they generalize from web images to surveillance.
  2. Open-Set Evaluation: We implemented a "watch-list" protocol to measure how well systems reject unknown distractors, a critical requirement for forensic applications.
  3. Super-Resolution (SR) Integration: We investigated whether pre-processing blurry images with SR models (such as SRCNN or VDSR) improves recognition.

Key Findings

  • The "Surveillance Gap" is Massive: There is a startling drop in performance when moving from web photos to surveillance. The CentreFace model achieved a 29.9% Rank-1 rate on QMUL-SurvFace, compared to 65.2% on MegaFace in a similar closed-set test—a 54% performance drop.
  • Super-Resolution is Not a Silver Bullet: Surprisingly, current SR methods often have a negative or negligible effect on FR performance. While SR makes images look "sharper" to the human eye, it often fails to recover the specific discriminative features needed for automated recognition.
  • The Open-Set Tax: Adding distractors makes the task exponentially harder. In the realistic open-set challenge, the top-performing model achieved only a 13.8% success rate (at Rank-20) when the false alarm rate was capped at 10%.

Discussion and Future Directions

The results suggest that simply aligning resolution (e.g., down-sampling web images) does not solve the domain discrepancy. The noise patterns and sensor characteristics of real CCTV are fundamentally different from artificially degraded high-res photos.

Future research must focus on:

  • Domain Adaptation: Better techniques to bridge the gap between high-quality training data and low-quality target data.
  • Native SR: Training super-resolution models on native low-res/high-res pairs rather than artificial down-sampling.
  • Semantic Attributes: Utilizing mid-level attributes (gender, age, clothing) to boost recognition when facial details are missing.
  • Robust Alignment: Improving landmark detection on blurry, low-res faces to ensure better normalization.

The QMUL-SurvFace challenge is publicly available, inviting the community to tackle these critical, unsolved problems in practical surveillance.

Summary updated

Abstract

Face recognition (FR) is one of the most extensively investigated problems in computer vision. Significant progress in FR has been made due to the recent introduction of the larger scale FR challenges, particularly with constrained social media web images, e.g. high-resolution photos of celebrity faces taken by professional photo-journalists. However, the more challenging FR in unconstrained and low-resolution surveillance images remains largely under-studied. To facilitate more studies on developing FR models that are effective and robust for low-resolution surveillance facial images, we introduce a new Surveillance Face Recognition Challenge, which we call the QMUL-SurvFace benchmark. This new benchmark is the largest and more importantly the only true surveillance FR benchmark to our best knowledge, where low-resolution images are not synthesised by artificial down-sampling of native high-resolution images. This challenge contains 463,507 face images of 15,573 distinct identities captured in real-world uncooperative surveillance scenes over wide space and time. As a consequence, it presents an extremely challenging FR benchmark. We benchmark the FR performance on this challenge using five representative deep learning face recognition models, in comparison to existing benchmarks. We show that the current state of the arts are still far from being satisfactory to tackle the under-investigated surveillance FR problem in practical forensic scenarios. Face recognition is generally more difficult in an open-set setting which is typical for surveillance scenarios, owing to a large number of non-target people (distractors) appearing open spaced scenes. This is evidently so that on the new Surveillance FR Challenge, the top-performing CentreFace deep learning FR model on the MegaFace benchmark can now only achieve 13.2% success rate (at Rank-20) at a 10% false alarm rate.
BibTeX
@misc{cheng2018surveillance,
  author = {Zhiyi Cheng and Xiatian Zhu and Shaogang Gong},
  title = {{Surveillance Face Recognition Challenge}},
  year = {2018},
  eprint = {1804.09691},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1804.09691},
}
Figure