Abstract
This paper introduces an approach based on a generative model named Sparse Probabilistic Linear Discriminant Analysis (S-PLDA) in speaker verification. The model provides an alternative approach to deal with the non-Gaussian behavior of the latent variables, directly assuming they are based on Laplace prior. This distribution encourages the model to set many latent variables to zero. An expectation-maximization algorithm is derived to train model with a variational approximation to a range of heavy-tailed distributions whose limit is the Laplace. The variational approximation is also used to compute of likelihood ratio. This approach performed well on the tel-tel extended condition of the NIST 2010 Speaker Recognition Evaluation, and is competitive compared to the Gaussian Probabilistic Linear Discriminant Analysis, in terms of normalized Decision Cost Function and Equal Error Rate.
Introduction
In modern speaker verification, researchers utilize "i-vectors"—low-dimensional numerical representations of a person's voice. To determine if two i-vectors belong to the same individual, the industry standard has long been Gaussian Probabilistic Linear Discriminant Analysis (G-PLDA). However, G-PLDA assumes that the underlying latent variables defining a speaker follow a Normal (Gaussian) distribution. In practice, speech data often exhibits "heavy-tailed" or non-Gaussian behavior.
This paper proposes Sparse Probabilistic Linear Discriminant Analysis (S-PLDA), a generative model that replaces traditional Gaussian assumptions with Laplace priors. By encouraging sparsity in latent variables, S-PLDA aims to automatically ignore irrelevant dimensions in the latent space, making speaker representations more robust and efficient.
Sparse Probabilistic Linear Discriminant Analysis
The core innovation of S-PLDA is the shift from Gaussian to Laplace priors. Unlike a Gaussian curve, which tapers off quickly, a Laplace distribution features a sharp peak at zero. This mathematical property encourages the model to set many latent variables exactly to (or very close to) zero, a property known as sparsity. This effectively performs automatic feature selection, identifying which latent dimensions are truly useful for distinguishing speakers while filtering out noise.
Training a model with Laplace priors is mathematically challenging because standard equations lack closed-form solutions. The authors derived a specialized Expectation-Maximization (EM) algorithm using a variational approximation. This technique approximates the complex Laplace distribution with a family of heavy-tailed distributions that are computationally more tractable for parameter estimation and likelihood ratio scoring.
Experimental Setup
The authors evaluated the system using the NIST 2010 Speaker Recognition Evaluation (SRE) dataset, specifically the "tel-tel" core condition. Training data included a massive collection of speech from Switchboard II (Parts 2 & 3), Switchboard Cellular, and NIST SRE corpora from 2004 to 2008. The input features consisted of 36-dimensional vectors comprising 18 Mel-frequency cepstral coefficients (MFCCs) and their first derivatives.
Results and Discussion
The authors compared S-PLDA against the standard G-PLDA baseline using two primary metrics: Equal Error Rate (EER) and Minimum Decision Cost Function (minDCF).
A significant finding was S-PLDA's robustness to raw data fluctuations. While "length normalization" (scaling i-vectors to unit length) is common in speaker verification, S-PLDA provides a much larger performance boost when normalization is not used. In these conditions, S-PLDA showed a dramatic improvement; for male speakers, the EER dropped from 4.10% (G-PLDA) to 3.05% (S-PLDA), representing a relative improvement of approximately 25%.
Even with length normalization applied, S-PLDA maintained a competitive edge. For female speakers, the EER improved from 3.52% to 3.26%. Overall, S-PLDA consistently outperformed or matched G-PLDA across various dimensions of the "eigenvoice" space.
Conclusion
S-PLDA introduces a robust framework for speaker verification by leveraging sparsity to handle non-Gaussian data. By utilizing Laplace priors and a variational EM algorithm, the model effectively filters noise and focuses on discriminative speaker features. While variational methods increase computational complexity, the gains in error rates on standard benchmarks like NIST SRE 2010 demonstrate the value of this sparse approach. Future work may expand these techniques to other complex audio classification tasks beyond speaker verification.
Summary updated