Speaker Verification Using Neighborhood Preserving Embedding

Chunyan Liang, Jinchao Yang, Lin Yang, Yonghong Yan

TL;DR

This paper introduces Neighborhood Preserving Embedding (NPE) as a novel factor analysis technique for speaker verification. By preserving local neighborhood structures in speaker data, NPE creates a more discriminative low-dimensional representation, outperforming the current state-of-the-art total variability approach on the NIST SRE2008 dataset.

What Problem Does This Paper Address?

Speaker verification, the task of identifying an individual based on their voice, relies heavily on extracting discriminative features from speech signals. Traditional methods often use Gaussian Mixture Models (GMMs) and Support Vector Machines (SVMs). While techniques like Joint Factor Analysis (JFA) and Total Variability (TV) factor analysis have improved performance by modeling intersession variability (e.g., different recording conditions, channel effects), they primarily focus on global structures or separate speaker and channel information. This paper addresses the limitation of existing methods by proposing a technique that explicitly preserves the local neighborhood structure of speaker data, aiming to capture more nuanced speaker-specific information. The goal is to create a more robust and discriminative low-dimensional representation for speaker verification.

Methods

The core contribution is the application of Neighborhood Preserving Embedding (NPE) for speaker verification. NPE is a dimensionality reduction technique that aims to preserve the local manifold structure of the data. Unlike Principal Component Analysis (PCA) which focuses on global structure, NPE builds an adjacency graph where nodes represent speaker supervectors. Edges connect supervectors from the same speaker, and weights are assigned based on minimizing a specific objective function. This process leads to a generalized eigenvector problem that yields the NPE projection matrix.

The proposed system integrates NPE with existing techniques:

  • GMM Supervectors: Speaker utterances are represented as GMM supervectors, which are then adapted using MAP estimation.
  • PCA Projection: Before applying NPE, a PCA projection (specifically, Probabilistic PCA with EM algorithm) is used to reduce the dimensionality, similar to the total variability approach. This step captures global information.
  • NPE Projection: Following PCA, the NPE projection is applied to preserve local neighborhood information. The final transformation matrix is a combination of PCA and NPE matrices.
  • Intersession Compensation: Linear Discriminant Analysis (LDA) and Within Class Covariance Normalization (WCCN) are applied to the NPE-projected vectors to further compensate for intersession variability.
  • SVM Classifier: The final low-dimensional, compensated vectors are used as input to an SVM classifier for speaker verification.

Datasets

The experiments were conducted on the NIST 2008 Speaker Recognition Evaluation (SRE) corpus, specifically the telephone-telephone (tel-tel) condition. The paper mentions using Switchboard II, Switchboard Cellular, and telephone data from NIST SRE 2004, 2005, and 2006 for training. The NIST SRE 2004, 2005, and 2006 data sets were also used for training the NPE, WCCN, and LDA matrices. No new benchmarks were introduced; the focus was on evaluating against existing methods on a standard dataset.

Key Insights and Ideas

  • Local Structure Matters: The paper strongly suggests that preserving the local neighborhood structure of speaker data is crucial for improving speaker verification performance, going beyond the global structure captured by PCA or the separate factor modeling of JFA.
  • NPE as a Complement to PCA: Combining PCA (for global structure) with NPE (for local structure) creates a more comprehensive and discriminative low-dimensional representation.
  • Supervised Learning Advantage: NPE's supervised nature, utilizing labeled speaker information during training, allows it to effectively learn speaker-specific embeddings, which is a key advantage over unsupervised methods.
  • Effective Intersession Compensation: The proposed NPE features are shown to benefit from subsequent intersession compensation techniques like LDA and WCCN, further enhancing robustness.

Key Results

The proposed NPE method consistently outperformed the state-of-the-art total variability approach across various experimental setups:

  • Without Intersession Compensation: NPE achieved relative improvements of up to 24.1% in Equal Error Rate (EER) and 19.0% in minimum Decision Cost Function (minDCF) for females, and 18.9% in EER and 19.0% in minDCF for males.
  • With LDA Compensation: NPE + LDA showed relative improvements of up to 20.5% in EER and 15.5% in minDCF for males, and 20.1% in EER and 14.7% in minDCF for females compared to TV + LDA.
  • With WCCN Compensation: NPE + WCCN yielded additional gains of 15.7% in EER and 5.5% in minDCF for males, and 16.3% in EER and 11.2% in minDCF for females compared to TV + WCCN.
  • With Both LDA and WCCN: NPE + LDA + WCCN achieved the best performance, with EERs of 4.11% (male) and 5.52% (female), and minDCFs of 0.205 (male) and 0.252 (female).

Limitations and Future Work

  • Limited to Telephone Condition: The experiments were primarily conducted on the telephone-telephone condition of NIST SRE2008. The authors express interest in testing the NPE method on other acoustic conditions beyond telephone speech in future work.
  • Computational Complexity: While not explicitly detailed as a limitation, the multi-stage process involving graph construction, eigenvector solving, and subsequent compensation steps might imply higher computational costs compared to simpler methods.
  • Future Work: The authors suggest extending the evaluation to other acoustic conditions. Further exploration of the NPE algorithm's parameters and its interaction with different compensation techniques could also be fruitful.

Summary updated

Abstract

In this paper, we adopt a new factor analysis of neighborhood preserving embedding (NPE) for speaker verification under the support vector machine (SVM) framework. NPE aims at preserving the local neighborhood structure on the data and defines a low-dimensional speaker space called neighborhood preserving embedding space. We compare the proposed method with the state-of-the-art total variability approach on the telephone-telephone core condition of the NIST 2008 Speaker recognition evaluation (SRE) dataset. The experimental results indicate that the proposed NPE method outperforms the total variability approach, providing up to 24% relative improvement.
BibTeX
@inproceedings{liang2012speaker,
  author = {Chunyan Liang and Jinchao Yang and Lin Yang and Yonghong Yan},
  title = {{Speaker Verification Using Neighborhood Preserving Embedding}},
  booktitle = {Interspeech},
  year = {2012},
  url = {https://openreview.net/forum?id=2vlB6HvmUa},
}
Figure