DamoFD: Digging into Backbone Design on Face Detection

Yang Liu, Jiankang Deng, Fei Wang, Lei Shang, Xuansong Xie, Baigui SunICLR 2023 · OpenReview

Face detection (FD) has achieved remarkable success over the past few years, but these leaps often come at the cost of enormous computational requirements. When building lightweight face detectors for computation-scarce scenarios, this heavy cost severely limits real-world application.

To remedy this, several pioneering works have designed tiny face detectors using off-the-shelf Neural Architecture Search (NAS) technologies. However, these architectures are usually optimized for image classification tasks, making them sub-optimal for face detection. Presented at ICLR 2023, DamoFD introduces a framework that rethinks how we design the feature-extracting "backbones" of face detectors. By moving away from generic classification architectures and utilizing a mathematically grounded search proxy, DamoFD achieves state-of-the-art performance with incredibly low computational overhead.

The Problem: Classification vs. Detection Backbones

Most lightweight face detectors rely on backbones (like MobileNet or ResNet) originally designed for image classification. However, classification and detection are fundamentally different in two key ways:

  1. Stage-Level Importance: Classification primarily relies on the final, high-level feature map to determine what is in an image. Detection, on the other hand, requires high-quality features at every stage to catch objects of varying sizes—small faces are detected in early, high-resolution stages, while large faces are detected in later, low-resolution stages.
  2. Computational Waste: In a detection framework, the backbone often consumes the lion's share of the inference budget. If the backbone isn't optimized for the specific "geometry" and scale distribution of face data, valuable FLOPs are wasted.

Existing NAS methods attempt to solve this by training many candidate models to see which performs best. Unfortunately, this "blind" search is computationally expensive, often taking hundreds of GPU hours.

Methodology: The DDSAR-Score

To address the inefficiencies of traditional NAS, the authors propose the Distribution-Dependent Stage-Aware Ranking Score (DDSAR-Score). This acts as a "zero-shot" proxy—a method to rank the potential effectiveness of a network architecture without actually having to train it.

Linear Regions Theory and SAR-Score

The core idea is rooted in deep learning theory, which states that a ReLU network's expressivity is related to the number of "linear regions" it can divide its input space into.

However, the authors found a flaw in applying this directly: standard mathematical bounds for linear regions grow exponentially with network depth.

Exponential growth of linear regions across network stages

As shown in the table above, the number of linear regions explodes as the stage index ($i$) increases. This exponential growth makes deep stages look unfairly "better" than shallow ones. To fix this, the authors modified the formula to normalize these scores into a Stage-Aware Ranking (SAR) Score, making the expressive power of different stages mathematically comparable.

Filter Sensitivity and Distribution-Dependent Weighting

To further refine the search proxy, the authors introduced two additional components:

  • Filter Sensitivity: A term was added to account for kernel sizes and strides, ensuring the search algorithm doesn't just blindly pick the smallest, most "efficient" kernels that lack sufficient receptive fields for detection.
  • Distribution-Dependent Weighting: This is the framework's "secret sauce." The importance of each backbone stage is weighted based on the actual distribution of face sizes in the target dataset. If a dataset contains predominantly tiny faces, the early network stages responsible for those faces receive a higher priority during the architecture search.

Finally, an Evolutionary Search Algorithm is used to navigate the architecture space, leveraging the DDSAR-Score to instantly evaluate and select the best candidates.

Key Results

The authors conducted comprehensive experiments on the challenging Wider Face benchmark dataset, evaluating models at standard VGA resolution (640x480). The resulting DamoFD models (ranging from 0.5GF to 34GF) consistently outperformed previous state-of-the-art models like SCRFD:

  • Accuracy Boost: At the highly constrained 0.5GF (Giga-FLOP) scale, DamoFD improved the Average Precision (AP) on the "Wider Face Hard" set by +2.5% compared to SCRFD.
  • Parameter Efficiency: DamoFD-34GF achieves better accuracy than SCRFD-34GF while using roughly 36% fewer parameters (6.24M vs 9.80M).
  • Search Speed: By using mathematical intuition over brute force, the search process took only 1 GPU hour. This is a massive ~100x speedup compared to competing NAS-based detectors that often require over 100 GPU hours.

Limitations and Future Work

While DamoFD presents a significant leap forward in efficient detector design, there are areas for future exploration:

  • Search Space Constraints: The proxy is novel, but the search space itself is still composed of standard building blocks (Residual, Bottleneck, and MobileNet blocks). Future work could explore more exotic cell types.
  • Hardware Specifics: The paper focuses on FLOPs and parameter counts. While these are standard proxies for efficiency, they do not always translate perfectly to real-world latency on specific hardware (like DSPs or NPUs) due to memory access patterns.
  • Generalization: The authors focused specifically on face detection. A natural next step is applying the DDSAR-Score logic to general object detection (e.g., the COCO dataset) to see if the "stage-aware" benefits hold for non-face objects.

Summary updated

Abstract

Face detection (FD) has achieved remarkable success over the past few years, yet, these leaps often arrive when consuming enormous computation costs. Moreover, when considering a realistic situation, i.e., building a lightweight face detector under a computation-scarce scenario, such heavy computation cost limits the application of the face detector. To remedy this, several pioneering works design tiny face detectors through off-the-shelf neural architecture search (NAS) technologies, which are usually applied to the classification task. Thus, the searched architectures are sub-optimal for the face detection task since some design criteria between detection and classification task are different. As a representative, the face detection backbone design needs to guarantee the stage-level detection ability while it is not required for the classification backbone. Furthermore, the detection backbone consumes a vast body of inference budgets in the whole detection framework. Considering the intrinsic design requirement and the virtual importance role of the face detection backbone, we thus ask a critical question: How to employ NAS to search FD-friendly backbone architecture? To cope with this question, we propose a distribution-dependent stage-aware ranking score (DDSAR-Score) to explicitly characterize the stage-level expressivity and identify the individual importance of each stage, thus satisfying the aforementioned design criterion of the FD backbone. Based on our proposed DDSAR-Score, we conduct comprehensive experiments on the challenging Wider Face benchmark dataset and achieve dominant performance across a wide range of compute regimes. In particular, compared to the tiniest face detector SCRFD-0.5GF, our method is +2.5 % better in Average Precision (AP) score when using the same amount of FLOPs. The code is avaliable at https://github.com/ly19965/FaceMaas/tree/master/face_project/face_detection/DamoFD.
BibTeX
@inproceedings{liu2023damofd,
  author = {Yang Liu and Jiankang Deng and Fei Wang and Lei Shang and Xuansong Xie and Baigui Sun},
  title = {{DamoFD: Digging into Backbone Design on Face Detection}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=NkJOhtNKX91},
}
Figure