Face Anthropometry Aware Audio-visual Age Verification

Pavel Korshunov, Sébastien Marcel

TL;DR

This paper tackles the critical problem of automated age verification, especially for minors, by proposing a multimodal approach combining audio and visual cues. They introduce a novel "adaptive label distribution learning" (ALDL) strategy that leverages facial anthropometry for more accurate age estimation, particularly for children, and release a new video dataset annotated with age labels.

What Problem Does This Paper Address?

The increasing consumption of online content by children has raised significant concerns about their exposure to inappropriate or harmful material. Automated age verification is seen as a potential solution to enforce online safety regulations and protect minors. However, current age verification methods primarily rely on static images and often lack sufficient data for children, making them less effective. Furthermore, privacy concerns surrounding biometric data sharing limit the applicability of cloud-based solutions. This paper aims to address these limitations by developing robust, multimodal age verification techniques that can operate locally on devices and are more accurate, especially for younger age groups.

Methods

The authors explore age verification through three main modalities: image-based, audio-based, and multimodal (audio-visual).

  • Image-based: They utilize a MobileNetV2 architecture, pre-trained on face recognition tasks, as a backbone. They evaluate various training strategies including standard classification, regression, regression via classification (RVC), distribution learning, and their proposed Adaptive Label Distribution Learning (ALDL). ALDL dynamically adjusts the standard deviation of age labels during training based on facial anthropometry and skin degradation, aiming to better distinguish critical age boundaries (like 18 years) while tolerating larger errors in older age groups.
  • Audio-based: Two approaches are investigated: a simpler model using Mel-frequency cepstral coefficients (MFCCs) with stacked LSTMs, and a more advanced end-to-end model adapted from a speaker diarization system (pyannote).
  • Multimodal: Three fusion strategies are explored to combine audio and visual information:
  • Score Fusion: Scores from independently trained image and audio models are combined using an SVM.
  • Feature Fusion: Embeddings from the last layers of image and audio models are concatenated and fed into an SVM.
  • Co-joint Training: Image and audio models are jointly trained at the embedding level with a shared loss function.

Datasets

The paper utilizes several existing datasets and introduces a new one:

  • Image Datasets:
    • UTKFace [28]: A large dataset with precise age labels, spanning from infants to 116 years old.
  • APPA-REAL [1]: Contains images with both real and apparent age labels.
  • UnFGA [9]: Images collected from Flickr, intended to reflect real-world imaging conditions, with age labels as short intervals.
  • Audio Dataset:
    • TIDIGITS [14]: A dataset of short audio samples with reliable age information.
  • Newly Introduced Dataset:
    • IdiapVideoAge (ours): A video dataset created by the authors, containing over 4000 videos sourced from VoxCeleb2 and Google's Child Speech dataset. A significant portion (about half) of these videos are annotated with age labels for children, making it a valuable resource for multimodal age verification research.

Key Insights and Ideas

  • The Importance of Children's Age Data: The paper highlights the scarcity of age-labeled data specifically for children in existing datasets and emphasizes the need for such data to improve age verification for minors.
  • Facial Anthropometry for Adaptive Learning: The ALDL strategy is a key innovation. By linking the "blurriness" of age labels (represented by the standard deviation in distribution learning) to known facial growth and degradation patterns, the model becomes more sensitive to critical age distinctions, particularly for children.
  • Multimodality Offers Complementary Information: Combining audio and visual cues consistently improves age verification performance compared to single modalities, suggesting that these modalities capture different, yet complementary, age-related information.
  • Co-joint Training Shows Promise for Children: While advanced training strategies like ALDL perform well, the paper notes that co-joint training of audio and visual models can be particularly effective for improving accuracy in the challenging childhood and puberty age categories.

Key Results

  • Image-based: The proposed adaptive (ALDL) approach achieved the best performance among image-based models on the UTKFace dataset in a seven-age-category scenario, with an F1-score of 0.591. This outperformed standard classification, regression, RVC, and distribution learning methods.
  • Audio-based: The pyannote model demonstrated superior performance compared to the simpler MFCC+LSTM model, achieving an F1-score of 0.976 on the IdiapVideoAge dataset for a binary (18-year old) scenario.
  • Multimodal: Multimodal approaches generally outperformed single-modality methods. For instance, combining an image classification model with a simple audio model via co-joint training on IdiapVideoAge achieved an F1-score of 0.731, outperforming the individual image (0.728) and audio (0.614 for pyannote) models. The best multimodal performance was achieved by advanced training strategies like adaptive and distribution learning without complex fusion, reaching an F1-score of 0.750.

Limitations and Future Work

  • Challenging Age Categories: The paper acknowledges that puberty, adolescence, and early adulthood remain the most challenging age categories for accurate verification, as indicated by confusion matrices.
  • Audio Model Training: The authors note that the audio-based models performed poorer when trained from scratch without pre-training, unlike the image-based models. This suggests that leveraging pre-trained audio models or more extensive audio datasets could further improve performance.
  • Future Directions: The authors suggest that further research is needed to bring age verification to a practically useful state. This includes exploring more advanced multimodal fusion techniques and further refining adaptive learning strategies to improve accuracy across all age groups, especially for children. The release of the IdiapVideoAge dataset is intended to facilitate future research in this area.

Summary updated

Abstract

Protection of minors against destructive content or illegal advertising is an important problem, which is now under increasing societal and legislative pressure. The latest advancements in an automated age verification is a possible solution to this problem. There are however limitations of the current state of the art age verification methods, specifically, the lack of approaches focusing on video-based or even solely audio-based approaches, since the image domain is the one with the majority of publicly available datasets. In this paper, we consider the problem of age verification as a multimodal problem by proposing and evaluating several audio- and image-based models and their combinations. To that end, we annotated a set of publicly available videos with age labels, with a special focus on the children age labels. We also propose a new training strategy based on the adaptive label distribution learning (ALDL), which is driven by facial anthropometry and age-based skin degradation. This adaptive approach demonstrates the best accuracy when evaluated across several test databases.
BibTeX
@inproceedings{korshunov2022face,
  author = {Pavel Korshunov and Sébastien Marcel},
  title = {{Face Anthropometry Aware Audio-visual Age Verification}},
  booktitle = {ACM Multimedia},
  year = {2022},
  doi = {10.1145/3503161.3548434},
}
Figure