Speaker recognition on lossy compressed speech using the speex codec

A. R. Stauffer, A. D. LawsonInterspeech 2009

TL;DR

This paper investigates how lossy audio compression, specifically using the Speex codec, impacts speaker recognition systems. They found that while high compression levels degrade performance, Speex can be effectively used to compress audio for storage and transmission without significant loss, and even improve recognition on other compressed formats like GSM.

What Problem Does This Paper Address?

Modern communication relies heavily on compressed audio for everything from VoIP to streaming. However, most research on speaker recognition (SR) has focused on uncompressed or minimally processed audio, neglecting the widespread use of lossy compression. This creates a gap: how does the significant data reduction inherent in codecs like Speex affect our ability to identify speakers? Specifically, the authors want to understand:

  • The direct impact of Speex compression on SR accuracy.
  • How SR systems perform when the compression level of the test audio doesn't match the compression level used to train the speaker models (mismatch).
  • If it's possible to create SR models that are robust to varying compression levels.
  • Whether Speex can be used to improve SR performance on audio compressed by other codecs, like GSM and Vorbis.

Methods

The core of this research involves systematically compressing audio using the Speex codec at various quality levels and then evaluating the performance of a standard speaker recognition system.

  • Speaker Recognition System: They employed a Gaussian Mixture Model (GMM) and Universal Background Model (UBM) approach, a common and robust technique in speaker recognition. Features extracted were mel-weighted and delta cepstra.
  • Speex Compression: The Speex codec was used to compress audio at 11 different quality levels, ranging from 0 (most compressed) to 10 (least compressed), in addition to an uncompressed control. The speexenc and speexdec utilities were used for this process.
  • Experimental Conditions:
    • Compression Mismatch: Speaker models were trained on uncompressed audio and then tested on audio compressed at various Speex levels.
  • Matched Compression: Both training and testing audio were compressed to the same Speex quality level.
  • Model Conditioning: To address mismatch, they explored training models using combinations of audio compressed at different Speex levels (e.g., mixing levels 4 and 8) and also including uncompressed data.
  • Other Codecs: Speex-compressed models were evaluated on audio compressed by GSM and Vorbis codecs.

Datasets

  • Primary Dataset: 240 conversations, each approximately 120 seconds long, recorded using a Samson C01U Condenser USB microphone in a low-noise environment.
  • Speakers: 120 speakers, each participating in two separate conversation sessions (separated by weeks/months) to simulate realistic cross-session evaluations.
  • Data Preparation: For each session, approximately 60 seconds of target speaker speech was extracted, sampled at 8000 Hz, 16-bit PCM. The secondary speaker's audio was removed.
  • Training/Testing Split: The first session of each speaker's conversation was used for training speaker models, and the second session was used for testing.
  • Cross-Session Accuracy: The paper notes that initial tests on same-day sessions yielded 100% accuracy, but using the cross-session data (which is more realistic) reduced accuracy to 89.2%, forming the basis for their study.

Key Insights and Ideas

  • Speex is a viable compression tool for SR: The research demonstrates that Speex can significantly reduce audio file sizes (by up to 85%) with minimal impact on SR accuracy (less than 1% degradation) when high quality levels are maintained.
  • Compression mismatch is a significant challenge: Even slight compression can degrade SR performance, with accuracy dropping substantially as compression quality decreases.
  • Model conditioning can mitigate mismatch: By training models on a mix of different compression levels, researchers can create more robust SR systems that perform well across a range of compression scenarios.
  • Speex models can boost SR on GSM: Interestingly, using Speex-compressed models can actually improve speaker recognition accuracy on GSM-compressed audio compared to using uncompressed models. This is attributed to Speex's ability to better handle the characteristics of GSM compression.
  • Vorbis behaves differently: Unlike GSM, Vorbis-compressed audio was best handled by uncompressed models, suggesting that the psychoacoustic approach of Vorbis might remove speaker-discriminative information differently than CELP-based codecs like Speex.

Key Results

  • Compression Mismatch: Testing uncompressed models on Speex-compressed data showed accuracy dropping significantly below 90% for quality levels less than 9. At the highest compression (level 0), accuracy fell to just above 10%.
  • Matched Compression: When training and testing data were compressed to the same Speex level, accuracy improved dramatically, reaching 89.2% even at the highest compression level (level 10).
  • Model Conditioning: A composite model trained on Speex levels 4 and 8, combined with uncompressed data, showed strong performance across various compression levels, outperforming mismatched conditions.
  • Speex on GSM/Vorbis:
    • On GSM-compressed data, using Speex models (level 7) improved accuracy from 47% (with uncompressed models) to 75%.
  • On Vorbis-compressed data, uncompressed models performed best, with only a 1.7% drop from matched results, achieving 87.5% accuracy.

Limitations and Future Work

  • Vorbis Performance: The paper notes that Vorbis, a music-oriented codec, did not benefit from Speex model conditioning and performed best with uncompressed models. Further research is suggested to understand this behavior and explore its compression rates.
  • Real-time vs. Offline: While Speex is designed for real-time communication, the study primarily focused on offline compression and evaluation.
  • Limited Codec Comparison: The study focused on Speex, GSM, and Vorbis. Exploring other common codecs could provide a more comprehensive understanding.
  • Future Work: The authors suggest further investigation into the Vorbis coder and exploring how to optimize models for different compression techniques. They also highlight the potential for Speex to be used for storing speaker voice samples with models for verification purposes.

Summary updated

BibTeX
@inproceedings{stauffer2009speaker,
  author = {A. R. Stauffer and A. D. Lawson},
  title = {{Speaker recognition on lossy compressed speech using the speex codec}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure