Abstract
Face detection has witnessed immense progress in the last few years, with new milestones being surpassed every year. While many challenges such as large variations in scale, pose, appearance are successfully addressed, there still exist several issues which are not specifically captured by existing methods or datasets. In this work, we identify the next set of challenges that requires attention from the research community and collect a new dataset of face images that involve these issues such as weather-based degradations, motion blur, focus blur and several others. We demonstrate that there is a considerable gap in the performance of state-of-the-art detectors and real-world requirements. Hence, in an attempt to fuel further research in unconstrained face detection, we present a new annotated Unconstrained Face Detection Dataset (UFDD) with several challenges and benchmark recent methods. Additionally, we provide an in-depth analysis of the results and failure cases of these methods.
Introduction
While face detection has seen massive leaps in performance, current state-of-the-art (SOTA) models still struggle when faced with real-world environmental "noise." In the lab, face detection feels like a solved problem. However, in the "wild"—think maritime surveillance, outdoor security cameras, or mobile photos taken in a storm—performance plummets.
Existing benchmarks, while large, don't explicitly capture the specific types of degradations encountered in these scenarios. Most models are trained on relatively "clean" images. When these models encounter rain streaks, heavy snow, haze, or even water droplets on a camera lens, their feature maps become corrupted by artifacts they weren't designed to handle. This paper introduces the Unconstrained Face Detection Dataset (UFDD), a new benchmark specifically designed to test detectors against weather degradations, blur, and lens artifacts, revealing a significant performance gap that current training sets like WIDER FACE fail to bridge.
The UFDD Dataset
The UFDD contains 6,425 images with 10,897 annotations across seven specific challenges: Rain, Snow, Haze, Blur, Illumination, Lens Impediments, and Distractors (images with no faces or non-human faces to test false positive rates).
Methodology
The researchers evaluated four representative detectors: Faster R-CNN, SSH (Single Stage Headless), S3FD (Single Shot Scale-invariant Face Detector), and HR-ER (designed for tiny faces).
To see if models could be "taught" to handle these conditions, they created a Synthetic WIDER FACE dataset. They applied mathematical models to simulate rain (using Gaussian noise masks), snow (mixture models), blur (focus and motion kernels), and lens impediments (water droplets/dirt masks). Evaluation was conducted using both internal cross-validation on UFDD and external testing (training on WIDER FACE/Synthetic WIDER FACE and testing on UFDD).
Results and Analysis
The results highlight a significant generalization gap. Models that perform exceptionally well on standard benchmarks see their Mean Average Precision (mAP) drop significantly on UFDD. This suggests that "unconstrained" in current literature usually just means "varied pose and scale," not "varied environmental quality."
Key Findings:
- Haze and Lens Artifacts are the "Boss Levels": Among all conditions, haze and lens impediments caused the most severe performance degradation. These conditions introduce sudden discontinuities in frequency and contrast that traditional CNN filters are not robust against.
- Tiny Face Detectors are Surprisingly Robust: The HR-ER model, originally designed for very small faces, consistently outperformed more recent models. Techniques used to find low-resolution "tiny" faces (like using feature maps from earlier layers) naturally translate well to finding "blurry" or "degraded" faces.
- Synthetic Training Helps: Fine-tuning models on the synthetically degraded WIDER FACE dataset led to measurable improvements. For instance, HR-ER’s mAP improved from 0.742 to 0.762 after synthetic fine-tuning.
- The Distractor Effect: Including "distractor" images significantly lowered mAP scores across the board, highlighting that current models are prone to "hallucinating" faces in complex, noisy backgrounds.
Conclusion and Future Work
The UFDD benchmark reveals that environmental degradations remain a major hurdle for face detection. While synthetic data augmentation provides a performance boost, it is not a complete solution. Future research should focus on architectural changes to develop neural network layers that are inherently invariant to artifacts produced by rain, snow, and blur, rather than relying solely on extensive data augmentation.
Summary updated