A method for on-line speaker indexing using generic reference models

Soonil Kwon, Shrikanth S. NarayananInterspeech 2003 · DOI

TL;DR

This paper introduces an "on-line" speaker indexing method that can identify speaker changes and classify segments in real-time audio streams without prior knowledge of the speakers. It achieves this by using a pre-trained set of generic speaker models to bootstrap the process, which are then adapted to the specific speakers in the test data.

What Problem Does This Paper Address?

In multimedia applications like meeting archives or teleconferencing, it's crucial to know who spoke when. Traditional speaker indexing methods often require offline processing or prior knowledge of the speakers involved. This paper tackles the challenge of on-line speaker indexing, where decisions must be made sequentially using only the current and past audio data. This is particularly difficult because building accurate speaker models on the fly with limited data is prone to errors, and the number of speakers is often unknown beforehand.

Methods

The proposed on-line speaker indexing system operates in several stages:

  1. Audio Classification: The system first discriminates between speech and non-speech segments (music, noise, silence). This is crucial to focus subsequent processing on relevant audio.
  2. Speaker Change Detection: Using a "Localized Search Algorithm" (LSA), the system detects potential speaker changes. LSA employs a sliding window approach, initially with a larger shift for efficiency, and then a finer, more precise search using the Generalized Likelihood Ratio (GLR) test when a potential change is detected. This balances computational cost with detection accuracy.
  3. Clustering and Model Adaptation:
    • Generic Reference Models: A key innovation is the use of a pre-determined set of "generic reference models." These models are trained on data independent of the test speakers, acting as a starting point.
  4. Clustering: Segments identified by the change detection are then clustered using these generic models. The model from the reference set that best matches a segment is selected.
  5. Model Adaptation: The selected generic model is then adapted to become a speaker-specific model using the Maximum a Posteriori (MAP) scheme. This adapted model replaces the original generic model in the reference set for future use.

Datasets

  • Training Data (for Reference Models): Data from the Speaker Recognition Benchmark NIST Speech (1999) was used. The paper mentions 16 speakers (8 male, 8 female) were randomly selected for training these generic models.
  • Evaluation Data:
    • Test 1: Speaker Recognition Benchmark NIST Speech (1999) (approx. 488 seconds of speech after filtering). This corpus consists of telephone conversations.
  • Test 2: HUB-4 Broadcast News Evaluation English Test Material (1999) (approx. 3336 seconds of speech after filtering). This corpus is more challenging due to the presence of speech, music, and background noise.

Key Insights and Ideas

  • Bootstrapping with Generic Models: The core idea of using pre-trained, speaker-independent models to initialize the on-line process is a clever way to overcome the "cold start" problem of having no prior speaker information. This significantly improves initial clustering accuracy.
  • Adaptation for Personalization: The continuous adaptation of generic models to speaker-specific models allows the system to learn and refine its understanding of each speaker throughout the audio stream.
  • LSA for Balanced Detection: The Localized Search Algorithm offers a practical compromise between the computational cost of fine-grained speaker change detection and the need for accurate boundaries.
  • Impact of Data Quality and Speaker Change Rate: The results highlight that cleaner audio (telephone conversations) and a lower rate of speaker changes lead to significantly better indexing performance.

Key Results

The paper reports the following speaker indexing error rates:

  • Test 1 (NIST Speech): 3.47% error rate.
  • Test 2 (HUB-4 Broadcast News): 15.74% error rate.

The authors note that Test 1 performed much better due to less background noise and a lower speaker change rate compared to Test 2. They also provide a distribution of contiguous errors, showing that for Test 1, over half of the errors were less than 500ms, while Test 2 had a longer tail of errors, with some exceeding 10 seconds.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Optimal Reference Model Set: The current method of selecting reference models is based on random selection. Future work could explore more optimal strategies for building this set to ensure better coverage and reduce overlap between models. Determining the optimal number of models in the reference set is also an open question.
  • Robust Speaker Change Detection: While GLR is effective, further improvements in speaker change detection algorithms could enhance overall performance.
  • Advanced Model Adaptation: While MAP adaptation is used, exploring other adaptation techniques could potentially optimize both the speed and accuracy of model learning.

Summary updated

BibTeX
@inproceedings{kwon2003method,
  author = {Soonil Kwon and Shrikanth S. Narayanan},
  title = {{A method for on-line speaker indexing using generic reference models}},
  booktitle = {Interspeech},
  year = {2003},
  doi = {10.21437/eurospeech.2003-724},
}
Figure