A Generalization of PLDA for Joint Modeling of Speaker Identity and Multiple Nuisance Conditions

Luciana Ferrer, Mitchell McLarenInterspeech 2018 · OpenReview

Abstract

Probabilistic linear discriminant analysis (PLDA) is the leading method for computing scores in speaker recognition systems. The method models the vectors representing each audio sample as a sum of three terms: one that depends on the speaker identity, one that models the within-speaker variability and one that models any remaining variability. The last two terms are assumed to be independent across samples. We recently proposed an extension of the PLDA method, which we termed Joint PLDA (JPLDA), where the second term is considered dependent on the type of nuisance condition present in the data (e.g., the language or channel). The proposed method led to significant gains for multilanguage speaker recognition when taking language as the nuisance condition. In this paper, we present a generalization of this approach that allows for multiple nuisance terms. We show results using language and several nuisance conditions describing the acoustic characteristics of the sample and demonstrate that jointly including all these factors in the model leads to better results than including only language or acoustic condition factors. Overall, we obtain relative improvements in detection cost function between 5% and 47% for various systems and test conditions with respect to standard PLDA approaches.

Introduction

In speaker recognition, the objective is to verify if two audio samples originate from the same speaker. The current industry standard for scoring these comparisons is Probabilistic Linear Discriminant Analysis (PLDA). Standard PLDA decomposes an audio vector into three components: speaker identity, within-speaker variability (nuisance factors), and residual noise.

A critical limitation of standard PLDA is its assumption that nuisance factors are independent for every sample. In practice, samples often share statistical dependencies—for instance, multiple recordings from the same speaker might use the same microphone or be in the same language. Ignoring these "tied" conditions leads to suboptimal performance, particularly in multilingual or acoustically diverse environments.

Methodology: Joint PLDA (JPLDA)

To address these dependencies, we propose Joint PLDA (JPLDA), which generalizes the standard model to handle multiple nuisance conditions simultaneously.

The Model Architecture

JPLDA decomposes the sample vector into:

  1. Speaker Identity: The target signal.
  2. Multiple Nuisance Terms: Distinct additive terms for factors such as language, microphone type, noise, reverberation, and codec.
  3. Residual Noise: Unexplained variance.

The innovation lies in tying latent variables: samples sharing a specific label (e.g., "Russian language") share the same latent variable for that condition, allowing the model to better isolate the speaker's true identity.

Training and Scoring

Training is conducted via a heuristic iterative procedure rather than a full Expectation-Maximization (EM) algorithm. By iteratively subtracting the estimated effects of all conditions except one and training a simplified PLDA on the remainder, the model stabilizes in approximately 10 iterations. Crucially, while nuisance labels are required during training, they are not needed during testing; the scoring process marginalizes over potential condition matches between enrollment and test samples.

Experimental Setup

The model was evaluated using three front-end systems: traditional i-vectors, hybrid alignment systems, and modern deep-learning speaker embeddings.

  • Training Data: The PRISM dataset, augmented with simulated noise, room impulse responses, and various telecommunication codecs.
  • Evaluation Data:
    • Mixer: NIST Speaker Recognition Evaluation telephone data (2005–2010) across multiple languages.
  • LASRS: A bilingual corpus designed for cross-language and cross-device testing.

Key Results

Performance was measured using the Minimum Detection Cost Function (MinDCF). The results demonstrate that modeling language and acoustic conditions jointly is superior to modeling them in isolation.

MinDCF performance across different datasets and conditions. JPLDA mcrnl (purple) consistently outperforms standard PLDA (SPLDA, gold) and other variants.

As shown in the figures, JPLDA achieved relative improvements between 5% and 47% compared to standard PLDA. On clean data, language modeling provided the most significant boost. However, in noisy or distorted environments, modeling acoustic conditions (noise, reverb, etc.) became essential. The JPLDA variant considering both language and all acoustic conditions (JPLDA mcrnl) consistently performed as well as or better than models focused on only one factor.

Conclusion

JPLDA represents a significant advancement in speaker recognition scoring by accounting for the statistical dependencies of nuisance factors. By acknowledging that samples are often "tied" by language or recording environment, the model achieves substantial gains in robustness. Future work will explore automatic estimation of nuisance labels for unlabeled datasets and formal EM refinement for multi-condition modeling.

Summary updated

Abstract

Probabilistic linear discriminant analysis (PLDA) is the leading method for computing scores in speaker recognition systems. The method models the vectors representing each audio sample as a sum of three terms: one that depends on the speaker identity, one that models the within-speaker variability and one that models any remaining variability. The last two terms are assumed to be independent across samples. We recently proposed an extension of the PLDA method, which we termed Joint PLDA (JPLDA), where the second term is considered dependent on the type of nuisance condition present in the data (e.g., the language or channel). The proposed method led to significant gains for multilanguage speaker recognition when taking language as the nuisance condition. In this paper, we present a generalization of this approach that allows for multiple nuisance terms. We show results using language and several nuisance conditions describing the acoustic characteristics of the sample and demonstrate that jointly including all these factors in the model leads to better results than including only language or acoustic condition factors. Overall, we obtain relative improvements in detection cost function between 5% and 47% for various systems and test conditions with respect to standard PLDA approaches.
BibTeX
@inproceedings{ferrer2018generalization,
  author = {Luciana Ferrer and Mitchell McLaren},
  title = {{A Generalization of PLDA for Joint Modeling of Speaker Identity and Multiple Nuisance Conditions}},
  booktitle = {Interspeech},
  year = {2018},
  url = {https://openreview.net/forum?id=D4cEeTSrN2T},
}
Figure