TL;DR
This paper introduces MaskMTL, a novel deep multi-task learning framework designed to predict facial attributes (age, gender, ethnicity) from images where faces are partially occluded by masks. By leveraging hierarchical parameter sharing and non-local attention, MaskMTL achieves competitive accuracy on masked faces, addressing a critical challenge in the post-pandemic era for applications like contactless authentication.
What Problem Does This Paper Address?
The COVID-19 pandemic has made face masks a ubiquitous part of daily life. This widespread adoption poses a significant challenge for facial recognition and attribute prediction systems, which are crucial for various applications, including secure access control, personal device unlocking, and even public health monitoring (e.g., identifying vulnerable groups). Traditional facial analysis methods often rely on clear, unoccluded facial features. When faces are masked, these systems struggle to extract reliable information, leading to decreased accuracy. While previous research has tackled general face occlusion, the specific and widespread occlusion caused by masks requires dedicated solutions. This paper aims to bridge this gap by developing a robust method for predicting key facial attributes from masked images.
Methods
The core of the proposed solution is a deep Multi-Task Learning (MTL) architecture called MaskMTL. The authors employ a hierarchical parameter sharing strategy, which is a hybrid approach combining benefits of hard and soft parameter sharing. In this setup, a shared encoder learns common representations for all tasks, while task-specific layers are introduced for each attribute. This allows for efficient feature learning and regularization, reducing the risk of overfitting.
Specifically, the architecture consists of:
- A shared encoder: This part extracts general facial features.
- Auxiliary tasks: Age estimation and gender recognition are treated as auxiliary tasks. These tasks help regularize the main task.
- Main task: Ethnicity recognition is the primary focus.
- Non-local attention: Introduced after the encoder, this mechanism helps the network focus on visible facial features, which is crucial when parts of the face are obscured by a mask. It computes responses by considering the weighted sum of features from all positions, effectively highlighting relevant facial regions.
The model is trained using a combination of L1 loss for age, categorical cross-entropy for ethnicity, and binary cross-entropy for gender.
Datasets
The primary dataset used is the UTKFace Dataset. This dataset is chosen for its diversity in age, gender, and ethnicity (White, Black, Asian, Indian, Others), and its inherent variations in illumination, pose, occlusion, and scale. To create masked facial images, the authors utilize the MaskTheFace tool. This tool generates masks with various textures and colors, applied to faces with different poses, effectively simulating real-world masked scenarios. The UTKFace dataset is split into training (Part I), testing (Part II), and validation (Part III) sets.
Key Insights and Ideas
- Masked faces are a distinct domain: The authors acknowledge that simply augmenting masks onto existing datasets creates a new domain that requires specific model training, rather than relying solely on pre-trained models for unmasked faces.
- Hierarchical parameter sharing is effective for attribute correlation: By structuring the MTL with auxiliary tasks (age, gender) influencing the main task (ethnicity), the model learns more robust and correlated features, leading to better performance.
- Non-local attention is crucial for occluded faces: Focusing attention on visible facial regions significantly improves the model's ability to extract relevant information when masks obscure key features.
- Lightweight architecture can achieve competitive results: The proposed encoder uses only 0.3M parameters, significantly fewer than larger models like ResNet-18 (11M parameters), yet achieves comparable or better performance on masked faces.
Key Results
- Competitive accuracy on masked faces: The MaskMTL model achieves a gender accuracy of 0.8913, race accuracy of 0.7927, and an age L1-loss of 12.15 on the masked UTKFace dataset.
- Outperforms standard MTL methods: When compared to various standard MTL approaches (e.g., Fixed MTL, GradNorm, Pareto MTL) on the masked dataset, MaskMTL demonstrates superior or comparable performance.
- Improved performance with non-local attention: The inclusion of non-local attention is shown to further boost prediction accuracy.
- Qualitative analysis reveals challenges: The paper highlights that age prediction can be challenging for very smooth faces (e.g., young children) or blurred images. Gender prediction struggles with side profiles or ambiguous features. Ethnicity prediction can be confused by similar-looking races or poor image contrast.
Limitations and Future Work
- Low recall for "Others" ethnicity: The authors note that the model has a low recall for the "Others" ethnicity class, suggesting that the presence of similar attributes across different ethnicities makes it difficult to distinguish.
- Future work: The authors intend to explore learning other facial attributes beyond age, gender, and ethnicity. They also suggest that pre-training the model on a larger, unmasked face dataset could further improve performance.
Summary updated