A multiclass framework for speaker verification within an acoustic event sequence system

Nicolas Scheffer, Jean-François BonastreInterspeech 2006

TL;DR

This paper introduces a novel "multiclass" approach to speaker verification by breaking down speech into smaller "class events" and analyzing sequences within these events. This method, called C-AES, aims to capture more nuanced, speaker-specific information than traditional whole-utterance analysis, leading to improved performance when combined with existing systems.

What Problem Does This Paper Address?

Traditional speaker verification systems often rely on modeling the entire speech utterance, typically using Gaussian Mixture Models (GMMs). While effective, these methods can overlook subtle, speaker-specific patterns that might be more evident when analyzing smaller segments of speech. The authors propose that analyzing sequences within distinct acoustic classes (like vowels or fricatives) can reveal more discriminative information than a global analysis. This is particularly relevant as datasets grow and the need for more robust and fine-grained speaker identification increases.

Methods

The core of the paper is the C-AES (Class-Acoustic Event Sequence) framework. It builds upon an existing Acoustic Event System (AES) which first extracts "Feature Events" (FEs) from speech using a GMM. The novelty lies in generating a second set of events called "Class Events" (CEs) by clustering these FEs. This creates a lower-resolution representation of the speech signal.

The C-AES system then applies an AES analysis independently to each CE. This means that instead of analyzing the entire sequence of FEs, the system analyzes sequences of FEs within each CE. The information from these multiple, class-specific analyses is then combined. A key technical contribution is the modification of the TFLLR (Term Frequency Log Likelihood Ratio) kernel to incorporate "apriori knowledge" of these classes, allowing for a more informed scoring process. This apriori knowledge can be estimated either by simple frequency counts or a more sophisticated Maximum A Posteriori (MAP) estimation.

Datasets

The experiments were conducted using the NIST SRE 2005 database. The training data for the background model (GMM/UBM) was derived from NIST SRE 1999 and 2002 databases. The evaluation was performed on the NIST SRE 2005 "all trial set (det1)" for male speakers, consisting of 280 speakers with approximately 2.5 minutes of speech per utterance.

Key Insights and Ideas

  • Intraclass analysis is discriminative: The paper strongly suggests that analyzing sequences within specific acoustic classes (CEs) provides more speaker-specific information than analyzing the entire utterance globally.
  • Multiclass approach captures nuance: By breaking down speech into smaller, defined segments (CEs), the C-AES system can potentially capture finer details of a speaker's vocal characteristics.
  • Apriori knowledge is crucial: The performance of the C-AES system is significantly impacted by how the "class probabilities" are estimated. Using apriori knowledge (either simple frequency or MAP estimation) is essential for good performance.
  • Fusion is powerful: While C-AES alone might not outperform a standard AES, its real strength lies in its ability to be effectively fused with other systems, particularly GMM/UBM, leading to substantial performance gains.

Key Results

  • The C-AES system, when using MAP estimation for class probabilities, achieved a Detection Cost Function (DCF) of 5.87% and an Equal Error Rate (EER) of 13.89%.
  • When fused with a standard GMM/UBM system, the C-AES approach resulted in a 12% relative gain at the DCF and a 7% relative gain at the EER compared to a baseline GMM/UBM system.
  • The fusion of C-AES with GMM/UBM achieved the best performance with a DCF of 3.16% and an EER of 7.96%.
  • The paper highlights that a system without any class weighting performed significantly worse (DCF 9.80%, EER 28.5%) than systems that incorporated class probabilities.

Limitations and Future Work

  • Dictionary Size: The paper notes that the dictionary size for Feature Events was fixed at 128, and for Class Events at 8. The optimal sizes and the impact of varying these parameters are not fully explored.
  • Fusion Strategy: While fusion with GMM/UBM showed promising results, the paper mentions using a simple arithmetic mean for combining scores. More sophisticated fusion techniques could potentially yield further improvements.
  • Future Work: The authors propose extending the framework to a "multi-resolution" approach by generating more segmentation sets and feature sets at different resolutions. This would allow for combining multiple analyses of varying granularities into a single vector for the classifier, potentially capturing even richer speaker information.

Summary updated

BibTeX
@inproceedings{scheffer2006multiclass,
  author = {Nicolas Scheffer and Jean-François Bonastre},
  title = {{A multiclass framework for speaker verification within an acoustic event sequence system}},
  booktitle = {Interspeech},
  year = {2006},
}
Figure