Using a Kind of Novel Phonotactic Information for SVM Based Speaker Recognition

Xiang Zhang, Hongbin Suo, Qingwei Zhao, Yonghong YanIEICE Trans. Information and Systems 2009 · OpenReview

Abstract

In this letter, we propose a new approach to SVM based speaker recognition, which utilizes a kind of novel phonotactic information as the feature for SVM modeling. Gaussian mixture models (GMMs) have been proven extremely successful for text-independent speaker recognition. The GMM universal background model (UBM) is a speaker-independent model, each component of which can be considered as modeling some underlying phonetic sound classes. We assume that the utterances from different speakers should get different average posterior probabilities on the same Gaussian component of the UBM, and the supervector composed of the average posterior probabilities on all components of the UBM for each utterance should be discriminative. We use these supervectors as the features for SVM based speaker recognition. Experiment results on a NIST SRE 2006 task show that the proposed approach demonstrates comparable performance with the commonly used systems. Fusion results are also presented.

Introduction

Traditional speaker recognition systems primarily utilize short-term acoustic features, such as Mel-frequency cepstral coefficients (MFCCs). While effective, these features often overlook "high-level" speaker characteristics, specifically phonotactics—the unique patterns and frequencies with which a speaker produces specific phonetic sounds. Capturing this information typically requires a full Automatic Speech Recognition (ASR) system to transcribe speech into phones or words, a process that is computationally intensive and sensitive to transcription errors. This paper proposes a method to extract phonotactic information directly from a Universal Background Model (UBM) framework, bypassing the need for a dedicated speech recognizer.

Methodology

The core innovation is the creation of a phonotactic supervector derived from a standard GMM-UBM. The process treats the $M$ Gaussian components of the UBM as proxies for underlying phonetic sound classes.

Schematic diagram of the proposed phonotactic feature extraction process.

The extraction involves the following steps:

  1. Posterior Alignment: For each frame in an utterance, the posterior probability for every Gaussian component is calculated, indicating how well a "sound class" matches that frame.
  2. Averaging: Instead of adapting Gaussian means (the standard approach), the system averages these posterior probabilities across the entire utterance. This results in a single vector $b$ of length $M$.
  3. Supervector Construction: This vector $b$ functions as a histogram of sound class usage. If Speaker A utilizes a specific sound class more frequently than Speaker B, their respective supervectors will reflect this difference.
  4. SVM Modeling: These supervectors serve as features for a Support Vector Machine (SVM) classifier. To maintain efficiency, a linear kernel is employed, and posteriors are calculated only for the top $C$ scoring mixtures per frame.

Experimental Setup

The approach was evaluated on the NIST 2006 Speaker Recognition Evaluation (SRE) core task (single-side 1-conversation training and testing). The UBM was trained using 1,744 conversation sides from 340 speakers across NIST 01, 02, 04, and 05 corpora. The acoustic feature set consisted of 36-dimensional MFCCs (12 coefficients plus first and second-order derivatives).

Results and Analysis

A key insight of this work is the unsupervised nature of the phonotactic extraction. By treating UBM components as "pseudo-phones," the system captures speaker-specific usage patterns without requiring transcripts.

Average posterior probabilities for two speakers across different utterances, demonstrating intra-speaker consistency and inter-speaker discriminability.

As shown in the visualization above, the posterior histograms are remarkably consistent across different utterances from the same speaker while remaining distinct between different speakers. This visual consistency underscores the discriminative power of the phonotactic supervector.

In terms of performance, the proposed system achieved an Equal Error Rate (EER) of 9.04% on the NIST SRE 2006 task. While slightly lower than the GSV-SVM baseline (7.37%), the system provides highly complementary information. When fused with traditional GMM and GSV-SVM systems, the performance improved significantly:

  • Baseline (GMM + GSV-SVM): 7.06% EER
  • Fused System (Baseline + Proposed): 6.47% EER

This represents an 8.4% relative improvement in EER and a 9.3% improvement in the minimum Decision Cost Function (minDCF), confirming that the frequency of sound usage captures speaker traits missed by traditional acoustic modeling.

Conclusion

The proposed phonotactic supervector offers a computationally efficient way to incorporate high-level speaker information into recognition systems. While its standalone performance is slightly below state-of-the-art acoustic models, its value as a complementary feature set is significant. Future work may explore refining UBM training to better define "sound classes" or optimizing the supervector construction to further enhance discriminability.

Summary updated

Abstract

In this letter, we propose a new approach to SVM based speaker recognition, which utilizes a kind of novel phonotactic information as the feature for SVM modeling. Gaussian mixture models (GMMs) have been proven extremely successful for text-independent speaker recognition. The GMM universal background model (UBM) is a speaker-independent model, each component of which can be considered as modeling some underlying phonetic sound classes. We assume that the utterances from different speakers should get different average posterior probabilities on the same Gaussian component of the UBM, and the supervector composed of the average posterior probabilities on all components of the UBM for each utterance should be discriminative. We use these supervectors as the features for SVM based speaker recognition. Experiment results on a NIST SRE 2006 task show that the proposed approach demonstrates comparable performance with the commonly used systems. Fusion results are also presented.
BibTeX
@article{zhang2009using,
  author = {Xiang Zhang and Hongbin Suo and Qingwei Zhao and Yonghong Yan},
  title = {{Using a Kind of Novel Phonotactic Information for SVM Based Speaker Recognition}},
  journal = {IEICE Trans. Information and Systems},
  year = {2009},
  url = {https://openreview.net/forum?id=sdslwtBPTH},
}
Figure