Multi-sample fusion with constrained feature transformation for robust speaker verification

Ming-Cheung Cheung, Kwok-Kwong Yiu, Man‐Wai Mak, Sun‐Yuan Kung2004 · DOI

TL;DR

This paper introduces a novel speaker verification system that fuses multiple speech samples from the same speaker by adaptively weighting their contributions. By combining this adaptive fusion with a "constrained stochastic feature transformation" technique to reduce channel noise, the system achieves a significant 22% reduction in equal error rate compared to traditional methods.

What Problem Does This Paper Address?

Speaker verification systems aim to confirm a person's identity based on their voice. A common challenge is that the quality of speech can vary significantly due to different microphones, phone lines, or background noise (channel distortion). Traditional systems often average scores from multiple speech samples from the same person. However, this "equal-weight" approach treats all samples equally, potentially diluting the impact of high-quality samples and being overly influenced by noisy ones. This paper tackles the problem of improving the robustness of speaker verification against channel distortions by developing a more intelligent way to combine multiple speech samples.

Methods

The core of the proposed method lies in two key components:

  • Multi-Sample Fusion with Adaptive Weights: Instead of simply averaging scores from different speech samples, this paper proposes assigning a unique weight to each sample's score. These weights are not fixed but are dynamically calculated based on how much a sample's score deviates from a speaker-specific reference score (obtained during enrollment). This allows the system to give more importance to samples that are more reliable or representative of the speaker.
  • Constrained Stochastic Feature Transformation (CSFT): To further combat channel distortion, the authors integrate CSFT. This technique works by transforming the speech features from a test utterance to better match the characteristics of the speaker and background models used during enrollment. Crucially, CSFT is "unsupervised," meaning it doesn't require explicit information about the type of channel or handset used, making it more practical for real-world scenarios. It achieves this by learning the statistical differences between the distorted test features and a composite model of the speaker and background.

Datasets

  • Training/Development: The paper mentions using speech data from 60 speakers in a development set to estimate prior scores and variances for the fusion weights.
  • Evaluation: The primary evaluation was conducted using the 2001 NIST speaker recognition evaluation set. This dataset contains speech from 174 target speakers and 60 development speakers, with a large number of gender-matched verification trials.

Key Insights and Ideas

  • Adaptive weighting is crucial: Simply averaging scores from multiple samples isn't optimal. By making fusion weights dependent on score reliability, the system can leverage high-quality samples more effectively.
  • Unsupervised channel compensation is powerful: CSFT's ability to reduce channel mismatch without needing explicit channel information is a significant advantage for real-world applications where handset types are unknown.
  • Two-level optimization works: Combining feature-level transformation (CSFT) with score-level fusion (adaptive weighting) provides a synergistic effect, leading to better performance than either technique alone.

Key Results

The proposed multi-sample fusion with CSFT demonstrated significant improvements over baseline methods on the 2001 NIST evaluation set:

  • Equal Error Rate (EER): The proposed system achieved an EER of 9.34%, representing a 22% relative reduction compared to the equal-weight fusion baseline (12.03% EER). It also outperformed the Z-norm baseline (10.39% EER) by a notable margin.
  • Minimum Detection Cost (DCF): The system achieved a 16% improvement in minimum DCF compared to the equal-weight baseline and a 10% improvement compared to Z-norm.
  • The paper also notes that their approach achieved a lower minimum detection cost (0.0402) than a previously proposed short-time Gaussianization method (0.0440).

Limitations and Future Work

  • Limitations: The paper acknowledges that the CSFT technique is applied to the entire test utterance, which is then split into segments for fusion. This might not be ideal for very long utterances where channel conditions could change significantly within the utterance.
  • Future Work: While not explicitly detailed, the success of this two-level optimization approach suggests that further exploration into more sophisticated feature transformation techniques and adaptive weighting strategies could yield even greater improvements in speaker verification robustness. The paper also implicitly points towards the need for real-time implementation and testing on more diverse and challenging datasets.

Summary updated

Abstract

This paper proposes a single-source multi-sample fusion approach to text-independent speaker verification. In conventional speaker verification systems, the scores obtained from claimant’s utterances are averaged and the resulting mean score is used for decision making. Instead of using an equal weight for all scores, this paper proposes assigning a different weight to each score, where the weights are made dependent on the difference between the score values and a speaker-dependent reference score obtained during enrollment. Because the fusion weights depend on the verification scores, a technique called constrained stochastic feature transformation is applied to minimize the mismatch between enrollment and verification data in order to enhance the scores’ reliability. Experimental results based on the 2001 NIST evaluation set show that the proposed fusion approach outperforms the equal-weight approach by 22% in terms of equal error rate and 16% in terms of minimum detection cost.
BibTeX
@misc{cheung2004multi,
  author = {Ming-Cheung Cheung and Kwok-Kwong Yiu and Man‐Wai Mak and Sun‐Yuan Kung},
  title = {{Multi-sample fusion with constrained feature transformation for robust speaker verification}},
  year = {2004},
  doi = {10.21437/interspeech.2004-522},
}

Loading related content...

Figure