TL;DR
This paper introduces AgeFormer, a novel video-based approach for age classification that leverages spatio-temporal body dynamics, outperforming traditional face-centric methods, especially in challenging real-world scenarios like occlusions and blur. To support this, they also release P-Age, a new video dataset specifically designed for age classification in these difficult conditions.
What Problem Does This Paper Address?
Estimating a person's age from visual data is a long-standing challenge with applications ranging from social robotics to surveillance. Traditional methods primarily rely on facial features, which are susceptible to various real-world issues like low resolution, poor lighting, and, crucially, occlusions (e.g., masks, blur, or simply not facing the camera). These face-centric approaches falter when the face isn't clearly visible, limiting their practical utility. The paper argues that a single frame of a face is insufficient and proposes a shift towards video-based analysis that considers the entire body's dynamics to overcome these limitations and preserve privacy.
Methods
The core of the proposed solution is the AgeFormer architecture, a two-stream network designed to capture both spatial and temporal information.
- Main-Stream (Video-based): This stream utilizes TimeSformer, a convolution-free video transformer, to learn spatio-temporal features from the entire body. It processes sequences of frames, capturing how body parts (limbs, torso, etc.) move and change over time. This is crucial for understanding dynamics beyond static facial features.
- Supporting-Stream (Image-based): This stream employs EfficientNet B2, a powerful and efficient 2D CNN, to extract features from aligned face crops. This stream acts as a complementary module, providing detailed facial information when available.
- Fusion: Features from both streams are fused using a multi-head attention (MHA) module. Specifically, the main-stream's features act as the "Query," while the supporting-stream's features serve as the "Key" and "Value." This allows the model to selectively attend to relevant facial information based on the broader body context. A skip connection and a final linear layer with softmax activation are used for age classification into four groups.
Datasets
- P-Age Dataset (New): Introduced by the authors, this dataset is scraped from Pexels (a royalty-free stock footage website). It consists of 4,568 downloaded videos, filtered down to 1,672 videos, focusing on age classification into four groups: Baby/Toddler, Adolescent, Adult, and Elderly. A key characteristic is that many videos feature occluded or blurred faces, simulating real-world privacy-preserving scenarios.
- P-Age-Face Dataset: A face-only variant derived from P-Age, created by sampling frames and extracting face crops. This allows for direct comparison with existing face-based methods.
- Benchmark Datasets: The model is also cross-tested on existing video datasets like Charades, Smarthome, and Thumos-14 to evaluate its generalizability, even without training on them.
Key Insights and Ideas
- Body Dynamics as Key Age Cues: The paper strongly advocates that body dynamics (posture, limb movement, etc.) provide significant, often overlooked, cues for age estimation, especially when faces are obscured.
- Video Transformers for Spatio-Temporal Reasoning: TimeSformer's ability to capture long-range spatio-temporal dependencies is shown to be highly effective for age estimation, moving beyond frame-level analysis.
- Complementary Strengths of Face and Body: The two-stream architecture effectively combines the detailed information from faces (when available) with the broader contextual understanding from body dynamics, leading to more robust predictions.
- Privacy Preservation Through Body Focus: By shifting the focus from faces to the entire body, the method inherently supports privacy-preserving age estimation in sensitive applications.
- Dataset Gap: The creation of the P-Age dataset highlights a gap in existing benchmarks, which are largely face-centric and don't adequately represent challenging real-world conditions.
Key Results
- SOTA Performance on P-Age: AgeFormer achieves state-of-the-art (SOTA) results on the proposed P-Age dataset, outperforming other video-based action recognition architectures adapted for age classification (Table 3).
- Superiority over Face-Based Methods: On the P-Age-Face dataset, AgeFormer (even when using only its face-stream) and its full video-based version significantly outperform existing SOTA face-based age classifiers (Table 4).
- Robustness to Occlusions and Blur: The model demonstrates strong performance even on blurred or blacked-out faces (Table 5), validating the importance of body dynamics.
- Cross-Dataset Generalization: AgeFormer shows promising performance on unseen datasets like Smarthome and Charades (Figure 5), indicating good generalizability.
Limitations and Future Work
- Confusion in Adult/Teen Classes: The model sometimes confuses the "Teen" and "Adult" classes, particularly in difficult cases (Figure 6a).
- Low-Light and Severely Degraded Video: Performance can degrade in extremely low-light conditions or with severely degraded video quality (Figure 5).
- Potential for Improvement with Face Features: The authors suggest that if a facial crop is achievable, leveraging the facial feature context from the supporting stream could further resolve ambiguities, especially in cases where the apparent age is borderline.
- Future Directions: The success of AgeFormer in age classification opens doors for exploring similar video-based approaches for other demographic classifications (gender, ethnicity) and for more complex video understanding tasks.
Summary updated