A fast speaker indexing using vector quantization and second order statistics with adaptive threshold computation

Konstantin BiatovInterspeech 2010

TL;DR

This paper introduces a faster, two-stage approach to speaker indexing by combining Vector Quantization (VQ) for an initial quick screening with the more computationally intensive Bayesian Information Criterion (BIC) for final merging. They also propose an on-line method to adaptively determine the crucial threshold parameter without needing separate development data, significantly speeding up the process while maintaining accuracy.

What Problem Does This Paper Address?

Speaker indexing, the task of grouping audio segments by speaker, is crucial for organizing large audio archives and enabling real-time applications like speaker retrieval. Existing state-of-the-art methods, often relying on sophisticated statistical models like the Bayesian Information Criterion (BIC), are computationally expensive, making them impractical for large-scale or real-time scenarios. A key challenge is efficiently identifying which segments to merge while accurately distinguishing between speakers. Furthermore, these methods typically require a manually tuned threshold parameter, which is sensitive to feature choices and data characteristics, necessitating separate development datasets for optimization.

Methods

The authors propose a two-stage agglomerative clustering algorithm:

  1. Fast Initial Screening (VQ): Instead of comparing all possible pairs of segments using BIC directly, they first extract MFCC features and estimate Gaussian Mixture Models (GMMs) for the entire audio data. The means of these GMMs are used as codebook centroids for Vector Quantization (VQ). Each audio segment is then represented by a histogram of its VQ codebook indices. The distance between segments is calculated using cosine distance on these histograms. This VQ-based step quickly identifies a smaller set of "N-closest" candidate segments for merging.
  2. Precise Merging (BIC): For the N-closest candidate pairs identified in the first stage, the more computationally expensive BIC is applied to make the final merging decision. This significantly reduces the number of BIC calculations needed.

Additionally, they introduce an on-line threshold computation method. This approach estimates the optimal threshold parameter ($\lambda$) by analyzing the ABIC (Adjusted BIC) values derived from splitting each segment into two parts. This allows the threshold to adapt to the specific data and features being used, eliminating the need for external development data.

Datasets

The experiments were conducted using 10 audio files from the ESTER corpus, a French national project for broadcast news transcription. This corpus contains a large number of speakers and audio segments. The data was preprocessed to a 16kHz sampling rate and 16-bit resolution, with features consisting of MFCCs and energy.

Key Insights and Ideas

  • VQ as a "Fast Match" for BIC: The core idea of using VQ as a computationally cheap pre-filter to narrow down BIC comparisons is a clever way to drastically reduce computational load without sacrificing too much accuracy.
  • Adaptive Thresholding is Crucial: The paper highlights that the BIC threshold is highly data-dependent. Their on-line, data-driven approach to threshold estimation is a significant contribution, making the system more robust and easier to deploy without extensive tuning.
  • Two-Stage Approach Balances Speed and Accuracy: The combination of a fast, approximate method (VQ) followed by a precise, albeit slower, method (BIC) is a common and effective strategy in many computational tasks. Here, it's applied effectively to speaker clustering.

Key Results

  • Significant Speedup: The proposed VQ-BIC method achieved a speedup of approximately 5 times compared to the baseline BIC-only approach and was more than 20 times faster than real-time.
  • Comparable Accuracy: Despite the speedup, the speaker purity (SP) and cluster purity (CP) metrics (both segment and frame level) remained comparable to the state-of-the-art BIC-based clustering. For instance, segment-level purity was around 0.84-0.85 for the VQ-BIC method, very close to the BIC baseline's 0.831/0.805.
  • Effective Thresholding: The on-line threshold computation method demonstrated its effectiveness across different feature sets, providing a data-dependent $\lambda$ value that resulted in good performance.

Limitations and Future Work

  • Dependence on N: The performance and speedup of the VQ-BIC method are dependent on the choice of 'N' (the number of closest candidates). While they tested various values, an optimal strategy for selecting 'N' might be further explored.
  • Empirical Constants for Thresholding: The on-line threshold calculation uses empirically determined constants ($\alpha=2, \beta=0.5$) for combining the average $\lambda$ and its standard deviation. Further investigation into optimizing these constants or developing a more principled way to derive them could be beneficial.
  • Future Work: The authors suggest that the proposed method could be extended to more complex GMMs or other distance measures beyond BIC for the second stage. They also imply that the on-line thresholding technique could be applied to other speaker verification/identification tasks.

Summary updated

BibTeX
@inproceedings{biatov2010fast,
  author = {Konstantin Biatov},
  title = {{A fast speaker indexing using vector quantization and second order statistics with adaptive threshold computation}},
  booktitle = {Interspeech},
  year = {2010},
}
Figure