The estimation and kernel metric of spectral correlation for text-independent speaker verification

Eryu Wang, Kong-Aik Lee, Bin Ma, Haizhou Li, Wu Guo, Li-Rong Dai

Introduction

In text-independent speaker verification, the objective is to authenticate a speaker's identity irrespective of the spoken content. Traditional state-of-the-art systems, primarily based on Gaussian Mixture Models (GMMs) and Support Vector Machines (SVMs), have historically relied on "mean super-vectors." These vectors represent the average spectral characteristics of a speaker's voice. However, such systems often assume diagonal covariance, effectively ignoring the correlations between different frequency bands—known as spectral correlation.

This paper, presented at INTERSPEECH 2010, argues that while mean vectors capture the "location" of acoustic features, the "shape" and "correlation" of those features (captured in covariance matrices) provide a powerful, complementary signal. These correlations reflect unique physiological traits of the speaker's vocal tract. The authors propose a framework to estimate and compare these high-dimensional covariance structures to enhance verification performance.

Methodology

The core innovation lies in moving from a "homoscedastic" model (a single global correlation for all acoustic states) to a "heteroscedastic" model, where each component of the speaker's voice model possesses its own unique correlation structure.

Covariance Super-matrix and Estimation

The authors introduce the "covariance super-matrix," formed by concatenating full covariance matrices from a GMM into a massive block-diagonal matrix. To address the challenge of estimating these matrices from limited speech data, they employ Maximum A Posteriori (MAP) estimation. This technique adapts a Universal Background Model (UBM) to a specific speaker, ensuring stable matrix estimation even with short utterances.

Kernel Metrics for SVM

To compare these high-dimensional matrices within an SVM framework, two novel kernel metrics were introduced:

  1. Frobenius Angle (FA): A geometric approach that treats matrices as vectors and calculates the angle between them, incorporating a normalization step to mitigate noise-induced variance.
  2. Log-Euclidean (LE) Metric: This maps matrices into a logarithmic domain where standard Euclidean distances are mathematically valid. This serves as a computationally efficient approximation of the Kullback-Leibler (KL) divergence.

To further refine the data, Nuisance Attribute Projection (NAP) was utilized for channel compensation, removing variability introduced by different recording environments or communication channels.

Experimental Evaluation

The proposed methods were evaluated using the NIST 2006 Speaker Recognition Evaluation (SRE) core task, involving 810 target speakers and over 51,000 trials. Training for background models and SVMs was conducted on NIST SRE 2004, while NIST SRE 2005 data was used for score normalization (t-norm).

Results and Discussion

The experimental results highlighted the significant impact of modeling spectral correlations. Transitioning from a homoscedastic to a heteroscedastic view yielded a substantial performance gain.

Table 1: Performance comparison of homoscedastic vs. heteroscedastic models and the effect of NAP.

As shown in Table 1, using unique covariance matrices per GMM mixture (hetero) improved the Equal Error Rate (EER) by approximately 29% compared to a single global matrix (homo) before noise compensation. The Log-Euclidean metric proved particularly effective when paired with NAP, achieving an EER of 3.36%.

The ultimate performance was achieved by fusing the proposed covariance-based system with the standard mean super-vector system.

Figure 1: Detection Error Tradeoff (DET) curves for the various systems and their fusion.

The fusion of mean super-vectors (MU) and Log-Euclidean spectral correlations (LE) resulted in an EER of 2.68%, representing a 10.36% relative improvement over the MU-only baseline (2.99% EER). This synergy confirms that spectral correlation provides distinct information not captured by mean vectors alone.

Conclusion

This research demonstrates that spectral correlations are a vital component of a speaker's identity. By leveraging heteroscedastic covariance modeling and specialized kernel metrics, significant improvements in speaker verification accuracy can be achieved. Future work will focus on reducing the computational overhead of full covariance matrices and exploring "kernel-level" fusion strategies to further exploit the relationship between mean and covariance features.

Summary updated

BibTeX
@inproceedings{wang2010estimation,
  author = {Eryu Wang and Kong-Aik Lee and Bin Ma and Haizhou Li and Wu Guo and Li-Rong Dai},
  title = {{The estimation and kernel metric of spectral correlation for text-independent speaker verification}},
  booktitle = {Interspeech},
  year = {2010},
  doi = {10.21437/interspeech.2010-139},
}
Figure