Face detection (FD) has achieved remarkable success over the past few years, but these leaps often come at the cost of enormous computational requirements. When building lightweight face detectors for computation-scarce scenarios, this heavy cost severely limits real-world application.
To remedy this, several pioneering works have designed tiny face detectors using off-the-shelf Neural Architecture Search (NAS) technologies. However, these architectures are usually optimized for image classification tasks, making them sub-optimal for face detection. Presented at ICLR 2023, DamoFD introduces a framework that rethinks how we design the feature-extracting "backbones" of face detectors. By moving away from generic classification architectures and utilizing a mathematically grounded search proxy, DamoFD achieves state-of-the-art performance with incredibly low computational overhead.
The Problem: Classification vs. Detection Backbones
Most lightweight face detectors rely on backbones (like MobileNet or ResNet) originally designed for image classification. However, classification and detection are fundamentally different in two key ways:
- Stage-Level Importance: Classification primarily relies on the final, high-level feature map to determine what is in an image. Detection, on the other hand, requires high-quality features at every stage to catch objects of varying sizes—small faces are detected in early, high-resolution stages, while large faces are detected in later, low-resolution stages.
- Computational Waste: In a detection framework, the backbone often consumes the lion's share of the inference budget. If the backbone isn't optimized for the specific "geometry" and scale distribution of face data, valuable FLOPs are wasted.
Existing NAS methods attempt to solve this by training many candidate models to see which performs best. Unfortunately, this "blind" search is computationally expensive, often taking hundreds of GPU hours.
Methodology: The DDSAR-Score
To address the inefficiencies of traditional NAS, the authors propose the Distribution-Dependent Stage-Aware Ranking Score (DDSAR-Score). This acts as a "zero-shot" proxy—a method to rank the potential effectiveness of a network architecture without actually having to train it.
Linear Regions Theory and SAR-Score
The core idea is rooted in deep learning theory, which states that a ReLU network's expressivity is related to the number of "linear regions" it can divide its input space into.
However, the authors found a flaw in applying this directly: standard mathematical bounds for linear regions grow exponentially with network depth.
As shown in the table above, the number of linear regions explodes as the stage index ($i$) increases. This exponential growth makes deep stages look unfairly "better" than shallow ones. To fix this, the authors modified the formula to normalize these scores into a Stage-Aware Ranking (SAR) Score, making the expressive power of different stages mathematically comparable.
Filter Sensitivity and Distribution-Dependent Weighting
To further refine the search proxy, the authors introduced two additional components:
- Filter Sensitivity: A term was added to account for kernel sizes and strides, ensuring the search algorithm doesn't just blindly pick the smallest, most "efficient" kernels that lack sufficient receptive fields for detection.
- Distribution-Dependent Weighting: This is the framework's "secret sauce." The importance of each backbone stage is weighted based on the actual distribution of face sizes in the target dataset. If a dataset contains predominantly tiny faces, the early network stages responsible for those faces receive a higher priority during the architecture search.
Finally, an Evolutionary Search Algorithm is used to navigate the architecture space, leveraging the DDSAR-Score to instantly evaluate and select the best candidates.
Key Results
The authors conducted comprehensive experiments on the challenging Wider Face benchmark dataset, evaluating models at standard VGA resolution (640x480). The resulting DamoFD models (ranging from 0.5GF to 34GF) consistently outperformed previous state-of-the-art models like SCRFD:
- Accuracy Boost: At the highly constrained 0.5GF (Giga-FLOP) scale, DamoFD improved the Average Precision (AP) on the "Wider Face Hard" set by +2.5% compared to SCRFD.
- Parameter Efficiency: DamoFD-34GF achieves better accuracy than SCRFD-34GF while using roughly 36% fewer parameters (6.24M vs 9.80M).
- Search Speed: By using mathematical intuition over brute force, the search process took only 1 GPU hour. This is a massive ~100x speedup compared to competing NAS-based detectors that often require over 100 GPU hours.
Limitations and Future Work
While DamoFD presents a significant leap forward in efficient detector design, there are areas for future exploration:
- Search Space Constraints: The proxy is novel, but the search space itself is still composed of standard building blocks (Residual, Bottleneck, and MobileNet blocks). Future work could explore more exotic cell types.
- Hardware Specifics: The paper focuses on FLOPs and parameter counts. While these are standard proxies for efficiency, they do not always translate perfectly to real-world latency on specific hardware (like DSPs or NPUs) due to memory access patterns.
- Generalization: The authors focused specifically on face detection. A natural next step is applying the DDSAR-Score logic to general object detection (e.g., the COCO dataset) to see if the "stage-aware" benefits hold for non-face objects.
Summary updated