A fast speaker adaptation method using aspect model

Seongjun Hahm, Akinori Ito, Shozo Makino, Motoyuki SuzukiInterspeech 2008

TL;DR

This paper introduces a novel speaker adaptation technique using Probabilistic Latent Semantic Analysis (PLSA) to model acoustic features. By leveraging an "aspect model" derived from PLSA, the authors demonstrate improved word recognition rates with significantly less adaptation data compared to traditional methods like EM, MAP, and MLLR.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems often struggle with variability introduced by different speakers, microphones, and acoustic environments. While speaker-independent (SI) models provide a baseline, they perform considerably worse than speaker-dependent (SD) models. Speaker adaptation aims to bridge this gap by fine-tuning an SI model with a small amount of data from a target speaker. The core challenge is efficiently adapting a large number of model parameters without requiring extensive adaptation data, which can be impractical. Existing methods like Expectation-Maximization (EM), Maximum A Posteriori (MAP), and Maximum Likelihood Linear Regression (MLLR) face limitations, especially when adaptation data is scarce.

Methods

The paper proposes using an "aspect model," derived from Probabilistic Latent Semantic Analysis (PLSA), for speaker adaptation. PLSA is a statistical technique for analyzing co-occurrence data, typically applied to text. Here, it's extended to continuous density Hidden Markov Models (HMMs) used in ASR.

The core idea is to use PLSA to decompose the acoustic model's mixture components into "aspects." These aspects represent underlying latent semantic structures within the acoustic features. Instead of directly adapting all the parameters of the original mixture models, the proposed method focuses on adapting the weighting of these aspects. This approach effectively reduces the number of free parameters that need to be estimated during adaptation.

The paper formulates the PLSA model in an acoustic space, where "documents" (d) are replaced by HMM states and "words" (w) are replaced by acoustic feature vectors (x). They derive E-step and M-step equations for this formulation, similar to standard PLSA but adapted for acoustic modeling. For adaptation, they propose both global and state-dependent weighting of these aspects.

Datasets

The experiments were conducted using the Korean isolated word database, KLE452.

  • Training Data: 35 speakers, phone-balanced 452 isolated words (1st utterances).
  • Adaptation Data: 1, 5, 10, 50, 100, 150, 200 words from 3 test speakers (2nd utterances).
  • Test Data: 452 words from the same 3 test speakers (1st utterances).

The paper does not introduce a new benchmark dataset but uses a standard one for evaluating speaker adaptation.

Key Insights and Ideas

  • Aspect Model for Parameter Reduction: The key insight is that an aspect model, derived from PLSA, can represent acoustic features more compactly. By focusing adaptation on the weighting of these aspects rather than all mixture component parameters, the number of parameters to be adapted is significantly reduced.
  • PLSA as Mixture Decomposition: PLSA is presented as a method for decomposing existing mixture models. This decomposition doesn't alter the original distribution but provides a new perspective for adaptation.
  • Improved Adaptation with Less Data: The aspect model approach shows superior performance, especially when adaptation data is limited (e.g., 1-50 words). This suggests that the latent aspects capture more robust speaker characteristics than directly adapting individual mixture components.
  • Weighting Aspects is Sufficient: Surprisingly, the authors found that adapting only the weighting of the aspects (P(z)) yielded better results than adapting all parameters, highlighting the effectiveness of this latent structure.

Key Results

The paper reports experimental results comparing their proposed PLSA-based aspect model adaptation with standard methods (EM, MAP, MLLR) on the Korean isolated word recognition task.

  • Global Weighting: For global weighting, the proposed method with 2, 3, or 4 aspect models consistently outperformed SI, EM, MAP, and MLLR, particularly with small amounts of adaptation data (1-50 words). For instance, with only 10 words of adaptation, the PLSA-based methods showed a noticeable improvement over MLLR and EM.
  • State-Dependent Weighting: Similar trends were observed with state-dependent weighting. However, the paper notes that for adaptation data exceeding 50 words, the performance of the proposed method did not significantly surpass other methods.
  • Quantitative Improvements: While specific word recognition rates (WRR) are presented in graphs, the text highlights that the proposed method achieved improved results using only weighting values, especially for small adaptation datasets. For example, Figure 6 shows that even with 2 aspect models, the PLSA-based approach achieves higher WRR than MLLR and EM for adaptation data sizes up to 50 words.

Limitations and Future Work

  • Limited Adaptation Data Performance: The paper acknowledges that while the proposed method excels with small adaptation datasets, its advantage diminishes when more adaptation data is available (over 50 words).
  • Database Size: The authors note that the number of speakers used in the database was "not enough relatively."
  • Future Work: The authors suggest future work should involve performing experiments on larger databases with more speakers and exploring various noisy environments to further validate and extend their findings. They also express interest in applying the aspect model to other areas of ASR.

Summary updated

BibTeX
@inproceedings{hahm2008fast,
  author = {Seongjun Hahm and Akinori Ito and Shozo Makino and Motoyuki Suzuki},
  title = {{A fast speaker adaptation method using aspect model}},
  booktitle = {Interspeech},
  year = {2008},
}
Figure