Predicting Likability of Speakers with Gaussian Processes

Dingchao Lu, Fei ShaInterspeech 2012 · OpenReview

TL;DR

This paper tackles the challenge of automatically predicting how "likable" a speaker sounds based solely on their voice. The authors leverage Gaussian Processes (GP) and introduce novel techniques for feature selection and semi-supervised learning, achieving state-of-the-art results on a challenging benchmark. This work opens doors for applications where voice quality significantly impacts communication effectiveness.

What Problem Does This Paper Address?

We often intuitively judge how pleasant or likable someone sounds based on their voice, but quantifying this subjective quality with machines is surprisingly difficult. While previous studies have explored acoustic correlates of attractiveness (e.g., pitch preferences), current computational models fall short. This paper addresses the gap in accurately predicting speaker likability from audio alone, a task with potential applications ranging from improving virtual assistants to enhancing communication for public speakers. They specifically focus on the Interspeech 2012 Speaker Trait Challenge data, which provides a benchmark for this problem.

Methods

The core of the paper's approach lies in Gaussian Processes (GP), a flexible non-parametric Bayesian framework for modeling data. GPs allow for automatic inference of model parameters and can capture complex relationships.

The authors introduce two key novelties:

  1. Sparse Gaussian Processes (S-GP): To combat the high dimensionality of acoustic features and prevent overfitting, they propose a greedy algorithm to select a sparse, yet informative, subset of features. This method is inspired by orthogonal matching pursuit and iteratively adds feature groups that yield the highest accuracy, leading to more parsimonious and effective models.
  2. Semi-supervised Learning with Kernel PCA: They leverage the learned covariance functions from GP models to derive new features. By applying Kernel PCA to a combined dataset (training, development, and unlabeled test data), they create a new feature space that captures underlying data structure, leading to improved performance.

Datasets

The primary dataset used is the Speaker Likability Database (SLD), provided for the Likability Sub-Challenge of Interspeech 2012. This dataset consists of recorded German telephone speech with likability labels.

They also applied their methods to the Pathology Sub-Challenge of the same event, which involves predicting the intelligibility of voices from Dutch patients undergoing treatment.

Key Insights and Ideas

  • Parsimony is Powerful: Using a carefully selected subset of features (sparse models) often leads to better prediction accuracy than using all available features, especially in high-dimensional settings. This suggests that not all acoustic cues are equally important for likability.
  • Feature Engineering Matters: Deriving new features from learned kernel functions (via Kernel PCA) significantly boosts performance, demonstrating the value of exploring the latent structure of the data.
  • Gender Differences in Predictability: The study found that likability of male voices is generally easier to predict than female voices. This suggests that acoustic cues for likability might manifest differently or be more pronounced in male speech.
  • Cross-Domain Applicability: The proposed GP-based methods, particularly the sparse and semi-supervised extensions, proved effective not only for likability but also for the related task of predicting voice intelligibility.

Key Results

The authors achieved state-of-the-art results on the Likability Sub-Challenge:

  • Their best system achieved 60.1% unweighted accuracy (UA) on the likability task, a 1.86% relative improvement over the published baseline of 59.0%.
  • For the Pathology Sub-Challenge, they reached 73.7% accuracy, a 6.92% relative improvement over the baseline of 68.9%.
  • Gender-specific models for likability showed that male-specific models generally achieved higher accuracies.

Limitations and Future Work

  • Gender Information: For the likability test set, gender information was not available. The authors addressed this by first predicting gender and then applying the appropriate gender-specific model. However, this adds an extra layer of potential error.
  • Computational Cost: While GP offers flexibility, it can be computationally intensive, especially for classification. The sparse GP approach helps mitigate this.
  • Future Work: The authors plan to extend their methods with other robust modeling techniques and explore more sophisticated ways to incorporate gender-specific modeling, especially when gender information is not readily available. They also suggest that a deeper analysis of the learned feature groups could provide further insights into the acoustic correlates of likability.

Summary updated

BibTeX
@inproceedings{lu2012predicting,
  author = {Dingchao Lu and Fei Sha},
  title = {{Predicting Likability of Speakers with Gaussian Processes}},
  booktitle = {Interspeech},
  year = {2012},
  url = {https://openreview.net/forum?id=hUBaMxRmu_o},
}
Figure