TL;DR
This paper proposes a novel system that combines head pose estimation with age estimation to improve accuracy and reliability when analyzing faces in videos. By filtering out frames with extreme head poses, the system achieves more precise age predictions, addressing a key limitation of existing methods that struggle with non-frontal faces.
What Problem Does This Paper Address?
Estimating a person's age from their facial image is a challenging task in computer vision. While significant progress has been made using deep learning on static images, real-world applications often involve videos where faces can appear at various angles. Existing age estimation models, typically trained on frontal faces, suffer from a substantial drop in performance when encountering non-frontal views. This is due to how facial features change with pose, leading to inconsistent and inaccurate age predictions in video streams. The paper aims to bridge this gap by developing a system that can robustly estimate age even when the face is not directly facing the camera.
Methods
The core of the proposed system is a two-pronged approach:
- Age Estimation: The authors employ Deep Regression Forests (DRFs), a method that combines the power of deep learning with the robustness of random forests. Specifically, they fine-tune a pre-trained VGG-16 convolutional neural network (CNN) on facial image datasets. The CNN extracts features, which are then fed into a forest of deep regression trees. Each tree learns to map facial features to age, and their combined predictions provide the final age estimate. This approach is chosen for its high precision on frontal facial images.
- Head Pose Estimation: To address the issue of non-frontal faces, a multiloss convolutional neural network (CNN) is used to estimate the head pose. This network is trained to predict three rotational angles (yaw, pitch, and roll). The use of multiple loss functions, including mean squared error and cross-entropy, helps improve accuracy.
The key innovation lies in how these two components are integrated. The system first estimates the head pose for each frame in a video. Then, it applies a degree threshold to the sum of the estimated head pose angles. Age estimation is performed only on frames where the head pose falls within this threshold (i.e., the face is relatively frontal). This "value refinement" step effectively filters out frames that are likely to yield inaccurate age predictions due to extreme angles.
Datasets
The paper utilizes several publicly available datasets:
- Cross-Age Celebrity Dataset (CACD): A large-scale dataset containing celebrity faces, used for age estimation. The authors used a cleaned subset of 18,171 images.
- Asian Face Age Dataset (AFAD): Contains Asian faces with accurate age and gender labels, also used for age estimation. A subset of approximately 60,000 images (ages 18-39) was used.
- 300W-LP: A dataset derived from the 300W landmark dataset, specifically augmented to include a wide range of head poses. This dataset was used for training the head pose estimation network.
The authors also constructed two new facial video datasets for evaluation, containing frames from twelve-minute videos of Asian and European subjects, respectively. These datasets were used solely for testing the age estimation performance in a video context.
Key Insights and Ideas
- Head Pose is Crucial for Age Estimation Accuracy: The experiments clearly demonstrate that age estimation accuracy significantly degrades for non-frontal faces. This highlights the importance of considering head pose as a critical factor in video-based age estimation.
- Filtering Non-Frontal Faces Improves Reliability: By selectively performing age estimation only on frames with head poses within a defined threshold, the system achieves more stable and accurate results. This simple yet effective filtering mechanism significantly reduces the variance in age predictions over a video.
- Combined System Outperforms Standalone Age Estimation: The proposed integrated system, which leverages both age and head pose estimation, consistently outperforms methods that only focus on age estimation, especially in video sequences where head pose varies.
Key Results
- Improved Accuracy on Frontal vs. Non-Frontal: The study found that age estimation accuracy was substantially better for frontal facial images compared to faces at different angles. For instance, on the AFAD dataset, the Mean Absolute Error (MAE) was 3.73 for frontal faces versus 4.97 for non-frontal faces.
- Optimal Threshold Identification: Through experiments with varying head pose thresholds, the authors identified 30 degrees as a good trade-off between maintaining a sufficient number of samples and achieving good performance.
- Reduced Variance in Video Age Estimation: When tested on facial video datasets, the proposed method achieved a lower MAE and, importantly, a reduced variance in age estimation compared to other state-of-the-art methods. For the Asian video dataset, the proposed method achieved an MAE of 5.12 and a variance of 3.50, outperforming other methods. For the European dataset, the MAE was 5.56 with a variance of 4.31. The variance was reduced by 0.62 and 1.53 on the Asian and European datasets, respectively, compared to the best previous methods.
Limitations and Future Work
The authors acknowledge that their method currently discards non-frontal frames. They suggest that future work could involve developing methods to calibrate non-frontal samples rather than simply ignoring them, which would make the system more widely applicable. Additionally, they plan to collect and annotate more facial images from videos to create a new, comprehensive database that could be used for training the entire system, potentially leading to even more robust and reliable results. They also note that there might be some video frames that do not meet their frontal view criteria, implying that a complete solution would need to handle all possible poses.
Summary updated