maskedFaceNet: A Progressive Semi-Supervised Masked Face Detector

Shitala Prasad, Yiqun Li, Dongyun Lin, Dong ShengWACV 2021

TL;DR

This paper introduces maskedFaceNet, a novel semi-supervised learning approach for detecting masked faces in real-world scenarios. By leveraging unlabeled data and a progressive training strategy, it significantly improves accuracy and efficiency, addressing the limitations of existing face detectors in the context of widespread mask usage.

What Problem Does This Paper Address?

The COVID-19 pandemic has made face mask detection a critical requirement for public safety and surveillance. However, existing face detection systems, primarily trained on datasets of unmasked faces, perform poorly when faced with masks, which act as significant occlusions. Collecting and annotating large datasets of masked faces is a time-consuming and expensive endeavor. This paper tackles the challenge of building an accurate and efficient masked face detector with limited labeled data, a common issue in many real-world applications where expert annotation is costly.

Methods

The core of the proposed solution is maskedFaceNet, a lightweight, single-stage, feed-forward object detector inspired by the SSD architecture. It utilizes a modified VGG-16 backbone with additional "assistant layers" designed to enhance feature representation for masked faces. The key innovation lies in its progressive semi-supervised learning (SSL) strategy:

  1. Phase 1 (Pre-training): The network is initially trained on a large, general face dataset (WIDER face) using a grid loss to learn robust face features and reduce false positives, especially for partially occluded faces.
  2. Phase 2 (Semi-Supervised Learning): The pre-trained model is used to generate "pseudo-labels" for a large corpus of unlabeled video data (collected during the COVID-19 pandemic). This phase leverages the model's own confidence scores to weigh the contribution of unlabeled data, effectively learning from a much larger, albeit noisy, dataset.
  3. Phase 3 (Fine-tuning): The model is then fine-tuned on the newly created, smaller, labeled masked face datasets. This phase incorporates a "progressive training" of pseudo-labeled data, where higher-confidence predictions are prioritized, further refining the model.

The loss function is a combination of classification and regression losses, with a specific focus on using smoothL1 loss for regression, which is found to be more suitable for masked face detection.

Datasets

The authors introduce two new masked face datasets:

  • MASK-face_v1: Annotates entire heads with bounding boxes for "face" and "mask."
  • MASK-face_v2: Annotates only faces, similar to standard face detection datasets.

These datasets were created from real-world scenes and are balanced for masked and non-masked faces. They also utilize existing datasets like FaceMaskDataset and African Masked Face Dataset for training and evaluation.

Key Insights and Ideas

  • Leveraging Unlabeled Data is Crucial: The paper strongly demonstrates that semi-supervised learning, by effectively utilizing readily available unlabeled data, can significantly boost performance, especially when labeled data is scarce.
  • Progressive Learning for Robustness: The multi-phase training strategy, starting with general face detection and progressively incorporating masked face data (both labeled and pseudo-labeled), allows the model to adapt to the specific challenges of mask occlusion.
  • Confidence-Weighted Pseudo-Labeling: Instead of simply using pseudo-labels, the authors incorporate the model's confidence in its predictions to guide the learning process, making the SSL phase more effective.
  • SmoothL1 Loss for Masked Faces: The study suggests that smoothL1 loss is a better choice for bounding box regression in masked face detection compared to other losses, likely due to its robustness to outliers introduced by mask occlusions.

Key Results

  • maskedFaceNet achieves state-of-the-art performance on the proposed MASK-face_v1 and MASK-face_v2 datasets, outperforming methods like Faster R-CNN and FaceBoxes by a significant margin (e.g., 4-6% improvement).
  • On the FaceMaskDataset, maskedFaceNet achieved the best mean Average Precision (mAP), surpassing even RetinaMask.
  • The proposed semi-supervised approach provides a substantial performance boost (2-3.3%) compared to training solely on limited labeled data.
  • The lightweight version, maskedFaceNet light, also achieves very competitive results, demonstrating its suitability for real-time applications.
  • The model achieves real-time performance, with the light version reaching 38 FPS on CPU for 640 input resolution.

Limitations and Future Work

  • Limited Recursion: The authors mention that a recursive process of re-generating pseudo-labels and re-training could further improve accuracy, but they limited it to one recursion for the scope of this paper.
  • Future Work: The authors plan to extend their work to masked face recognition and address incorrect masked face detection. They also aim to apply the progressive SSL hypothesis to other challenging tasks with limited data and annotation, such as industrial applications, and explore domain adaptation via semi-supervised learning.

Summary updated

BibTeX
@inproceedings{prasad2021maskedfacenet,
  author = {Shitala Prasad and Yiqun Li and Dongyun Lin and Dong Sheng},
  title = {{maskedFaceNet: A Progressive Semi-Supervised Masked Face Detector}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
}
Figure