TL;DR
This paper tackles the persistent challenge of detecting small faces in images by introducing a novel "Density Aware Feature Enrichment" (DAFE-FD) approach. Instead of solely relying on anchor design, DAFE-FD enriches feature maps using density estimation, inspired by crowd counting techniques, to better highlight small faces. This method shows significant improvements, especially in challenging "hard" scenarios, and complements existing detection strategies.
What Problem Does This Paper Address?
Detecting small faces in images remains a significant hurdle in computer vision. While recent advancements have focused on improving anchor box strategies and multi-scale detection to better capture objects of varying sizes, these methods often struggle with extremely small faces (e.g., 15x15 pixels). This difficulty arises from the inherent mismatch between the receptive fields of feature maps and the tiny objects, as well as the low overlap between predefined anchor boxes and these small ground truth faces. Existing approaches, while effective for larger objects, degrade in performance when faced with these diminutive targets.
Methods
The core innovation of DAFE-FD lies in its Density Estimator Module (DEM) and Feature Enrichment Module (FEM).
- Density Estimator Module (DEM): Inspired by crowd counting techniques, the DEM estimates a per-pixel density map of faces in an image. This module takes feature maps from the early convolutional layers of a backbone network (VGG-16 in this case) and, through a series of convolutions and upsampling, outputs a density map. This density map essentially highlights areas with a higher likelihood of containing faces, particularly useful for small and potentially occluded ones.
- Feature Enrichment Module (FEM): This module uses the output of the DEM to enhance the feature maps from an earlier convolutional layer (conv3) of the backbone network. Instead of simple concatenation or addition, which can be problematic due to differing feature map ranges and channel counts, the FEM directly uses the density map to modulate the conv3 features. Specifically, it adds a scaled version of the density map (replicated to match channel dimensions) to the conv3 feature maps. This process accentuates features corresponding to high-density areas (likely small faces) and suppresses others.
The enriched feature maps are then fed into a multi-scale detector framework. The proposed architecture uses multiple detectors (D1-D4) placed on different convolutional layers, allowing for detection across various scales. Each detector also incorporates a Context Aggregation Module (CAM), which uses atrous convolutions to increase receptive fields without significantly increasing computational cost, further aiding in capturing contextual information. The overall loss function is a multi-task loss combining classification, bounding box regression, and the density estimation loss.
Datasets
The paper uses the following datasets:
- WIDER: Used for training and extensive ablation studies. It's a large-scale dataset known for its diverse challenges, including variations in scale and occlusions.
- FDDB: Used for evaluation. The authors note that they use it for testing only and employ rectangular bounding boxes, unlike some other methods that use elliptical regression or train on FDDB.
- Pascal Faces: Used for evaluation. This dataset is a subset of the PASCAL person layout dataset.
Key Insights and Ideas
- Density as a Signal for Small Faces: The most significant insight is leveraging crowd density estimation, a technique proven effective for counting in dense and occluded scenes, as a direct signal to improve small face detection. The intuition is that areas with higher face density are precisely where small faces are most likely to be found and most difficult to detect.
- Feature Enrichment Over Anchor Design: While many recent methods focus on refining anchor box strategies, this paper argues for the importance of enhancing the feature representations themselves. The DEM and FEM directly address the "feature map deficiency" for small objects.
- Intelligent Feature Fusion: The paper highlights the limitations of simple feature fusion techniques (addition/concatenation) when combining features from different modules (like DEM and the backbone). The proposed FEM offers a more robust way to integrate this density information.
- Complementary Approach: DAFE-FD is presented as a method that can be easily integrated with existing anchor-design strategies, offering a way to further boost performance without discarding prior advancements.
Key Results
- WIDER Dataset:
- DAFE-FD achieved a significant improvement of 1.8% in mAP for "hard" faces on the WIDER validation set compared to a baseline with context aggregation.
- The proposed FEM contributed an overall improvement of 0.8% in mAP for hard faces over the baseline with context.
- On the WIDER test set, DAFE-FD outperformed existing state-of-the-art methods on the "hard" subset and achieved comparable or better performance on other subsets. For instance, DAFE-FD (single-scale) achieved better results than SSH-single-scale in all subsets, with a 1.8% improvement in the "hard" set.
- FDDB Dataset: DAFE-FD achieved consistently better performance in discrete scores and was comparable to other methods in continuous scores, despite not using elliptical regression or training on FDDB.
- Pascal Faces Dataset: The proposed DAFE-FD method outperformed existing methods like S3FD and Faceness.
- Computational Efficiency: The inference speed for DAFE-FD was measured at 50 msec/image for FDDB/PASCAL (400x800 resolution) and 190 msec/image for WIDER (1200x1600 resolution). The computational overhead introduced by the DEM was minimal (178 msec/image for baseline without DEM vs. 190 msec/image with DEM).
Limitations and Future Work
- Limitations:
- The paper notes that simple feature addition and concatenation of DEM outputs did not yield improvements, suggesting that the specific fusion mechanism (FEM) is crucial.
- While computational overhead is minimal, the DEM does add some processing time compared to a pure detection model.
- Future Work:
- The authors suggest that DAFE-FD is complementary to anchor design strategies and can be used in conjunction with them for further performance gains.
- The success of density estimation for small faces hints at potential applications in other detection tasks where small object detection is challenging.
Summary updated