Video Face Recognition System: RetinaFace-mnet-faster and Secondary Search

Qian Li, Nan Guo, Xiaochun Ye, Dongrui Fan, Zhimin TangarXiv 2020 · arXiv

Abstract

Real-time video face recognition faces significant challenges due to environmental noise, varying face scales, and high computational latency. This paper presents an optimized pipeline featuring a wavelet-based pre-processing module, a streamlined "RetinaFace-mnet-faster" detector, and an HNSW-powered secondary search mechanism. The proposed system achieves a 70.2% speedup on CPU and maintains high accuracy across standard benchmarks like LFW and WIDER FACE.

Introduction

While face recognition in static, high-quality images is largely a solved problem, "in the wild" video analysis remains difficult. Video frames often suffer from motion blur and poor lighting. Furthermore, faces in surveillance footage are frequently small, making standard detection inefficient. Finally, the latency of brute-force searching large databases for every frame is a major bottleneck for real-time deployment. This work aims to bridge the gap between academic accuracy and practical real-time performance.

System Architecture

The proposed system follows a multi-stage pipeline designed for efficiency and robustness. It begins with raw video frames, proceeds through pre-processing and detection, and concludes with a high-speed feature retrieval stage.

Overview of the video face recognition pipeline, from frame extraction to feature library matching.

Optimization Strategies

Image Pre-processing

To mitigate environmental noise, the system employs an Image Pre-processing Module. This module uses median filters for noise reduction and Wiener filters for restoration. A wavelet-based contrast enhancement technique is also applied to ensure the downstream detector receives high-quality input even from "fuzzy" scenes.

RetinaFace-mnet-faster

The authors optimized the light-weight RetinaFace (MobileNet-based) by "trimming the fat." Specifically, the stride-32 detection branch—typically used for large faces—was removed, as large faces are rare in the target video scenarios. The model focuses on strides 4 and 8 to better capture small-scale faces. Additionally, key point detection is disabled for the smallest faces, significantly reducing computational cost with minimal impact on alignment accuracy.

For the recognition phase, the system replaces brute-force matching with the Hierarchical Navigable Small World (HNSW) algorithm. HNSW constructs a multi-layered graph for rapid approximate nearest neighbor (ANN) searches.

Visualization of the HNSW search process, showing multi-layered graph traversal for rapid feature retrieval.

The retrieval is further refined by a "secondary search" mechanism. This approach uses logical operations between initial and subsequent search results to enhance precision, particularly in large-scale datasets where false positives are more likely.

Experimental Results

The system was evaluated on LFW, WIDER FACE, and a custom practical dataset of 15,000 faces.

  • Computational Efficiency: RetinaFace-mnet-faster achieved a 16.7% speedup on a Tesla P40 GPU and a 70.2% speedup on a single-threaded CPU compared to the baseline.
  • Retrieval Speed: The HNSW-based search was 82% faster than brute-force retrieval.
  • Accuracy: The system achieved 99.61% accuracy on LFW (with ArcFace) and 78.4% AP on the WIDER FACE "Hard" set, outperforming the baseline despite the increased speed.

Conclusion

By combining targeted architectural pruning with advanced graph-based retrieval, this work demonstrates that real-time video face recognition is achievable without sacrificing accuracy. Future work will focus on leveraging temporal consistency across frames to further improve robustness in extremely noisy environments.

Summary updated

Abstract

Face recognition is widely used in the scene. However, different visual environments require different methods, and face recognition has a difficulty in complex environments. Therefore, this paper mainly experiments complex faces in the video. First, we design an image pre-processing module for fuzzy scene or under-exposed faces to enhance images. Our experimental results demonstrate that effective images pre-processing improves the accuracy of 0.11%, 0.2% and 1.4% on LFW, WIDER FACE and our datasets, respectively. Second, we propose RetinacFace-mnet-faster for detection and a confidence threshold specification for face recognition, reducing the lost rate. Our experimental results show that our RetinaFace-mnet-faster for 640*480 resolution on the Tesla P40 and single-thread improve speed of 16.7% and 70.2%, respectively. Finally, we design secondary search mechanism with HNSW to improve performance. Ours is suitable for large-scale datasets, and experimental results show that our method is 82% faster than the violent retrieval for the single-frame detection.
BibTeX
@misc{li2020video,
  author = {Qian Li and Nan Guo and Xiaochun Ye and Dongrui Fan and Zhimin Tang},
  title = {{Video Face Recognition System: RetinaFace-mnet-faster and Secondary Search}},
  year = {2020},
  eprint = {2009.13167},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2009.13167},
}
Figure