Phonetic Variability Influence on Short Utterances in Speaker Verification

Ignacio Viñals, Alfonso Ortega, Antonio Miguel, Eduardo LleidaIberSPEECH 2018 · DOI

TL;DR

This paper tackles the performance drop in speaker verification systems when dealing with short audio clips. They propose a novel method to account for phonetic mismatch between enrollment and test utterances, leading to significant improvements, especially for short audio.

What Problem Does This Paper Address?

Speaker verification systems, which aim to confirm if an audio clip belongs to a specific speaker, typically perform best with longer audio samples. When presented with short utterances, their performance degrades significantly. This is a critical issue for applications like call center analysis or meeting transcription where short speech segments are common. The paper identifies that this degradation isn't just due to less data, but more importantly, due to the phonetic mismatch – the difference in the sounds spoken between the enrollment and test clips. Standard i-vector PLDA systems, while state-of-the-art for long utterances, don't explicitly account for this phonetic difference, leading to misinterpretations where phonetic variations are mistaken for speaker variations.

Methods

The core of the paper's contribution lies in quantifying and compensating for phonetic mismatch.

  • Phonetic Mismatch Metric: They introduce a metric based on the Kullback-Leibler (KL) divergence between the zeroth-order Baum-Welch statistics of the enrollment and test utterances. These statistics, derived from the GMM-UBM step, are related to the acoustic content and thus, indirectly, to the phonetic content of the speech. The KL divergence measures how different these acoustic profiles are.
  • Fusion with PLDA Score: This phonetic mismatch metric is then fused with the standard i-vector PLDA score. The proposed system uses logistic regression to combine the original PLDA score with the calculated KL distance. This fusion aims to adjust the final score based on how phonetically similar or dissimilar the enrollment and test utterances are. A higher phonetic similarity suggests a more reliable score, while a lower similarity warrants a more lenient score to account for potential mismatches.

Datasets

The experiments were conducted using subsets of the NIST SRE10 core-extended core-extended female dataset. The authors used long utterances (around 5 minutes) as a baseline and created two types of short utterances:

  • Chopped Short Utterances: Random segments of 3-60 seconds were extracted from the original long utterances.
  • Phonetically Balanced Short Utterances: Segments were selected to maintain a similar phoneme distribution to the original long utterances. This subset, while not naturally occurring, was crucial for isolating the impact of phonetic variability versus just data length.

Key Insights and Ideas

  • Phonetic Mismatch is Key: The paper strongly suggests that phonetic mismatch is a more significant contributor to performance degradation in short utterances than simply the reduced amount of data (estimation variability).
  • Standard Systems Invent Content: When faced with short utterances, standard i-vector PLDA systems tend to "invent" missing phonetic content. This leads to decisions based on unreal speech, making comparisons unreliable.
  • Phonetic Mismatch vs. Speaker Variability: The core problem is that current systems confuse phonetic variability (what is said) with speaker variability (who is speaking). This leads to increased False Negatives when phonetic content differs.
  • Simple Fusion Works: A straightforward fusion of the phonetic mismatch metric with the PLDA score yields significant improvements, demonstrating the value of explicitly considering phonetic information.

Key Results

  • Performance Degradation: Chopped short utterances showed a 163.69% relative degradation in EER compared to long utterances, while phonetically balanced short utterances showed a 26.46% degradation. This highlights the impact of phonetic variability.
  • Improvement with Fusion: The proposed compensated system achieved relative improvements of up to 10% in error rates for both short and long utterances.
  • Target Trial Improvement: The compensation specifically helped reduce errors in target trials (False Negatives), which were identified as the main source of degradation in short utterances.
  • Minimal False Positives: While the compensation improved performance, it led to a small increase in False Positive decisions, which were previously almost null.

Limitations and Future Work

  • Limitations: The paper acknowledges that the proposed phonetic mismatch metric (KL distance on Baum-Welch statistics) is a first step and might not capture all nuances of phonetic information. The fusion method is also a simple linear regression, suggesting more complex fusion strategies could be explored. The increase in False Positives is also a noted limitation.
  • Future Work: The authors suggest further research into more efficient and sophisticated ways to utilize phonetic information. This could involve exploring different phonetic feature extraction methods or more advanced fusion techniques to further refine the speaker verification system's performance with short utterances.

Summary updated

BibTeX
@inproceedings{vinals2018phonetic,
  author = {Ignacio Viñals and Alfonso Ortega and Antonio Miguel and Eduardo Lleida},
  title = {{Phonetic Variability Influence on Short Utterances in Speaker Verification}},
  booktitle = {IberSPEECH},
  year = {2018},
  doi = {10.21437/iberspeech.2018-2},
}
Figure