The use of subvector quantization and discrete densities for fast GMM computation for speaker verification

Guoli Ye, Brian MakInterspeech 2010

TL;DR

This paper introduces a new method called Subvector Quantized High-Density Discrete Model (SVQ-HDDM) to significantly speed up speaker verification systems that rely on Gaussian Mixture Models (GMMs). By cleverly quantizing feature vectors into smaller subvectors, they achieve an 8-25x speedup over traditional GMMs with minimal loss in accuracy, making real-time speaker verification more feasible.

What Problem Does This Paper Address?

Speaker verification systems often use Gaussian Mixture Models (GMMs) to model the acoustic characteristics of speakers and a universal background model (UBM). While effective, computing the likelihoods for GMMs can be computationally intensive, especially for large models. This slowness hinders real-time applications. Previous work attempted to speed this up by converting GMMs into High-Density Discrete Models (HDDM) using scalar quantization (SQ), which achieved a 30x speedup but at the cost of a 1.5% increase in Equal Error Rate (EER). The challenge is to achieve significant speedups without sacrificing verification accuracy.

Methods

The core innovation is the Subvector Quantized High-Density Discrete Model (SVQ-HDDM). This method builds upon the idea of discrete models but uses Subvector Quantization (SVQ) instead of Scalar Quantization (SQ).

Here's how it works:

  1. Partitioning: A d-dimensional acoustic feature vector is split into L smaller subvectors.
  2. Subvector Quantization (SVQ): Each subvector is then vector-quantized independently, creating an SVQ codebook for each partition.
  3. Full-Space Codeword Construction: A full-space VQ codeword is formed by combining codewords from each of the L subvector quantizers. This creates a discrete representation of the original d-dimensional space.
  4. HDDM Conversion: The probability mass function (pmf) of the SVQ-HDDM is derived by integrating the original GMM's probability density function (pdf) over the regions defined by these SVQ-constructed VQ codewords.

The paper also discusses practical considerations like:

  • Partitioning Strategy: For simplicity, they partition vectors into consecutive dimensions.
  • Subvector Dimension: A trade-off exists between subvector dimension, quantization error, storage, and search time.
  • Multi-stream SVQ-HDDM: To manage model size, they propose splitting the feature vector into independent "streams" (e.g., static and dynamic MFCCs) and modeling each stream with an SVQ-HDDM. This reduces the number of VQ bins significantly but sacrifices inter-stream correlation.

Datasets

  • NIST SRE 2001: Used for creating the GMM-UBM.
  • NIST SRE 2002: Used for creating speaker models (via MAP adaptation) and for evaluation. This dataset contains 1006 male utterances for UBM training and 139 male speakers for model adaptation. The evaluation involved 1,232 speaker trials and 14,630 impostor attempts.

Key Insights and Ideas

  • SVQ is more efficient than SQ: For the same number of bits, SVQ generally results in lower quantization error than SQ, leading to better accuracy. This is because VQ can capture correlations within a subvector, which SQ ignores.
  • SVQ-HDDM bridges the gap: SVQ-HDDM offers a sweet spot between the accuracy of full-space VQ (which can be slow) and the speed of SQ (which sacrifices accuracy).
  • Multi-stream modeling is a practical necessity: While splitting features into streams can reduce accuracy due to lost correlation, it's crucial for managing the potentially enormous size of SVQ-HDDM models.
  • SVQ-HDDM can match GMM performance: With sufficient bits, SVQ-HDDM can achieve accuracy comparable to the original GMM, a feat not possible with SQ-HDDM.

Key Results

  • Speedup: SVQ-HDDM provides an overall speedup of 8-25 times compared to traditional GMMs when speaker models are pre-loaded in memory.
  • Accuracy: SVQ-HDDM achieves performance very close to the parent GMM, with minimal loss in EER. For instance, SVQ-HDDM-g (with 3-dimensional subvectors) achieved an EER of 11.48%, very close to the 2-stream GMM's 11.68%.
  • Comparison to SQ-HDDM: For the same model size, SVQ-HDDM consistently outperforms SQ-HDDM.
  • Model Size: While larger than GMMs, the paper argues SVQ-HDDM model sizes are acceptable for many real-world applications.
  • Operating Characteristics: Figures 1 and 2 illustrate the trade-offs between EER and computation time for GMMs, SQ-HDDMs, and SVQ-HDDMs under different memory loading scenarios. SVQ-HDDM shows a clear advantage in speed for a given EER.

Limitations and Future Work

  • Model Size: The primary drawback of SVQ-HDDM is its larger model size compared to GMMs. While manageable with multi-stream approaches, it can still be substantial.
  • Memory Loading Time: When speaker models are loaded on-the-fly (not pre-loaded), the advantage of HDDMs over GMMs is significantly offset by the time it takes to load their larger models. The authors suggest that better server-grade hardware could mitigate this.
  • Loss of Correlation in Multi-stream: The multi-stream approach, while necessary for model size, inherently loses correlation between feature streams. The paper notes that the performance degradation due to this is comparable to using only the top-5 Gaussians in GMM likelihood computation.
  • Integration Precision: The paper mentions developing a custom integration program to balance speed and precision for calculating probabilities over SVQ codewords, implying that achieving perfect precision might be computationally prohibitive.

Summary updated

BibTeX
@inproceedings{ye2010use,
  author = {Guoli Ye and Brian Mak},
  title = {{The use of subvector quantization and discrete densities for fast GMM computation for speaker verification}},
  booktitle = {Interspeech},
  year = {2010},
}

Loading related content...

Figure