Age and gender classification using fusion of acoustic and prosodic features

Hugo Meinedo, Isabel TrancosoInterspeech 2010

TL;DR

This paper presents a novel approach to automatically classifying the age and gender of speakers in audio recordings by fusing multiple acoustic and prosodic features. Their system achieved significant improvements over baseline methods, demonstrating the power of combining diverse feature sets and classification strategies for robust paralinguistic analysis.

What Problem Does This Paper Address?

Accurately identifying a speaker's age and gender from their voice is a crucial task with applications ranging from multimedia retrieval to human-computer interaction. However, existing systems often struggle with variability in speech, limited training data for certain demographics (like children), and a lack of standardized evaluation procedures. This paper tackles the challenge of building more robust and accurate age and gender classifiers, specifically addressing the need to distinguish between multiple age groups (children, youth, adults, seniors) and three gender categories (child, female, male). The authors highlight the difficulty in classifying children's voices and the general challenge of achieving comparable results across different systems due to varying evaluation methods.

Methods

The core of the proposed system lies in a sophisticated fusion strategy. Instead of relying on a single model, the authors developed multiple "front-end" systems, each trained with different acoustic and prosodic features, classification algorithms, and training datasets.

  • Feature Extraction: They employed a variety of features, including:
  • Acoustic Features: Perceptual Linear Prediction (PLP) coefficients, energy, and modulation spectrogram features.
  • Prosodic Features: Pitch (f0) and its temporal variations.
  • Pre-computed Features: "arff450" features provided by the challenge organizers.
  • Classification Paradigms: A range of machine learning models were used, such as:
  • Support Vector Machines (SVM) with a linear kernel.
  • Multi-Layer Perceptrons (MLP) with varying architectures and input contexts.
  • Gaussian Mixture Models - Universal Background Model (GMM-UBM).
  • Fusion Strategy: The outputs from these diverse front-end systems were then combined using a "back-end" system. This back-end employed calibration and linear logistic regression (LLR) fusion to produce the final age and gender predictions. The motivation behind this multi-faceted approach is that the diversity of features, models, and data would lead to a more robust and accurate overall system.

Datasets

The researchers utilized four distinct datasets for training and evaluation:

  • aGender Corpus: Provided by the INTERSPEECH 2010 Paralinguistic Challenge, this corpus contains telephone speech from 795 speakers, labeled for age and gender. It was partitioned into training, development, and test sets.
  • CMU Kids Corpus: This corpus consists of sentences read by children aged 6-11, recorded in a controlled environment.
  • PF STAR Children Corpus: Provided by KTH Research, this corpus also features speech from children, recorded with both headset and desktop microphones, offering more speaker diversity.
  • BN ALERT Corpus: This corpus comprises European Portuguese Broadcast News recordings, labeled for gender but not age. It was used to augment the training data for gender classification.

All corpora were pre-processed, and most were down-sampled to 8kHz to match the aGender corpus's sampling frequency.

Key Insights and Ideas

  • Fusion is Key: The paper strongly emphasizes that combining multiple, diverse front-end systems significantly outperforms any single system. This highlights the benefit of leveraging different perspectives (features, models) on the same problem.
  • Data Augmentation Matters: Incorporating additional datasets, particularly for under-represented groups like children, led to substantial improvements in classification accuracy. This underscores the importance of comprehensive and balanced training data.
  • Feature Diversity is Crucial: The success of the system is attributed to the use of both short-term acoustic features (like PLP) and longer-term prosodic features (like pitch), as well as features capturing temporal dynamics.
  • Age Classification is Challenging: The results show that age classification, especially distinguishing between similar age groups (e.g., young vs. adult, adult vs. senior), is inherently more difficult than gender classification, leading to higher confusion rates.

Key Results

The fusion approach yielded impressive results:

  • Age Classification: The fused system achieved an absolute improvement of 4.1% in unweighted accuracy compared to the competition's baseline systems on the development set. The final competition result on the test set was 48.7% UA.
  • Gender Classification: The fused system demonstrated an even more significant improvement, achieving 5.8% absolute increase in unweighted accuracy over the baseline. The final competition result on the test set was 84.3% UA.
  • Individual Front-ends: The best individual front-end for age classification combined acoustic and prosodic features with an MLP on an expanded training set, achieving 49.2% UA. For gender, the best front-end combined acoustic and prosodic features with an MLP, using additional child and broadcast news data, reaching 82.2% UA.

Limitations and Future Work

  • Age Confusion: The authors acknowledge that confusions between adjacent age groups (Young/Adult, Adult/Senior) are expected due to the gradual nature of aging and the difficulty in establishing clear acoustic boundaries.
  • Gender Imbalance: For gender classification, they suspect that the excellent performance on male detection might be partly due to a larger proportion of male speakers in the training data.
  • Future Work: The paper mentions that the developed systems and the experience gained will be valuable for their participation in the European I-DASH project, suggesting continued research and application of these techniques. The development of a new age detection system, an area where they had no prior experience, is also highlighted as a significant outcome.

Summary updated

BibTeX
@inproceedings{meinedo2010age,
  author = {Hugo Meinedo and Isabel Trancoso},
  title = {{Age and gender classification using fusion of acoustic and prosodic features}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure