Bayesian speaker adaptation based on probabilistic principal component analysis

Dong Kook Kim, Nam Soo KimInterspeech 2000 · OpenReview

TL;DR

This paper introduces a novel Bayesian speaker adaptation method using Probabilistic Principal Component Analysis (PPCA) to create robust "canonical" speaker models. This approach effectively leverages prior knowledge from a large set of speakers to adapt a speech recognition system to a new speaker with very little data, outperforming existing methods.

What Problem Does This Paper Address?

Automatic speech recognition (ASR) systems often struggle when the speech they encounter differs from the data they were trained on. This mismatch can arise from various factors, including different speaking styles, accents, or even the specific characteristics of individual speakers. Speaker adaptation aims to quickly adjust an ASR system to a new speaker using a small amount of their speech, making the system more personalized and accurate. Traditional methods like Maximum A Posteriori (MAP) adaptation and Maximum Likelihood Linear Regression (MLLR) have been successful, but they can struggle with very limited adaptation data. The "eigenvoice" technique, which uses Principal Component Analysis (PCA) to find principal speaker characteristics, offers a way to incorporate prior knowledge but can be limited in its ability to fully converge to a true speaker-specific model. This paper tackles the challenge of achieving highly accurate speaker adaptation with minimal data by proposing a more principled way to learn and utilize prior speaker knowledge.

Methods

The core of this paper's contribution lies in its use of Probabilistic Principal Component Analysis (PPCA) for learning canonical speaker models. Unlike standard PCA, PPCA models the data as a linear combination of latent variables (the principal components) corrupted by Gaussian noise, and importantly, it provides a probabilistic framework. This means PPCA not only identifies the principal subspaces (eigenvoices) but also provides a prior distribution over these components.

The authors integrate PPCA into a Bayesian adaptation framework, specifically using the Maximum A Posteriori (MAP) criterion. The PPCA-derived canonical speaker models serve as the prior knowledge for the MAP adaptation. This allows the system to adapt to a new speaker by finding the most probable model parameters given both the prior knowledge and the new speaker's limited adaptation data. The adaptation formula derived from this approach effectively interpolates between the Maximum Likelihood (ML) estimate from the adaptation data and the PPCA prior estimate. This interpolation is crucial: when adaptation data is scarce, the PPCA prior has a stronger influence, leading to more stable adaptation. As more data becomes available, the ML estimate gains weight, and the adapted model converges towards the true speaker-specific model.

Datasets

The experiments were conducted using a Korean continuous digit recognition task.

  • Training Data: 105 speakers contributed to the initial training of speaker-independent (SI) models.
  • Evaluation Data: 35 different speakers were used for testing the adaptation methods.
  • Each speaker contributed 30-40 sentences, each containing 3-7 digits.

The paper does not introduce new benchmark datasets but rather uses existing data for evaluation.

Key Insights and Ideas

  • PPCA as a principled prior: PPCA offers a more statistically grounded way to learn prior speaker characteristics compared to standard PCA. By providing a probabilistic model, it allows for a more robust integration into Bayesian adaptation.
  • Unified adaptation framework: The proposed method elegantly combines the strengths of prior knowledge (from PPCA) and adaptation data (via MAP). This unified approach is key to achieving good performance with limited data.
  • Interpolation for robustness: The derived adaptation formula acts as a weighted average between the ML estimate and the PPCA prior. This interpolation is critical for handling low-resource adaptation scenarios, preventing overfitting to sparse data.
  • Convergence to true model: The Bayesian framework ensures that as more adaptation data is provided, the adapted model will converge to the true speaker-specific model, a desirable property for long-term adaptation.

Key Results

The paper reports significant improvements in word recognition rate (WRR) using the PPCA-based adaptation compared to conventional methods like MAP and MLLR, especially with limited adaptation data.

  • The SI system achieved a baseline WRR of 88.03%.
  • MAP and MLLR showed improvements over the SI system. For 2, 5, and 10 adaptation sentences, MAP achieved rates between 88.61% and 89.74%, while MLLR ranged from 88.15% to 88.60%.
  • The PPCA-based approach (and MLED, which uses PCA as a prior) consistently outperformed MAP and MLLR.
  • With 2 adaptation sentences, PPCA achieved rates between 89.24% and 89.55%.
  • With 5 adaptation sentences, PPCA achieved rates between 89.51% and 89.55%.
  • With 10 adaptation sentences, PPCA achieved rates between 90.09% and 90.21%.
  • Notably, for very sparse adaptation data (2 or 5 sentences), the PPCA-based method showed comparable or slightly better performance than MLED, and its performance was less sensitive to the chosen dimension (P) of the principal subspace. For 10 adaptation sentences, the PPCA approach showed a slight edge over MLED.

Limitations and Future Work

  • Dimensionality (P): While the paper explores varying the dimensionality (P) of the principal subspace, it notes that for very sparse adaptation data, the word recognition rate does not necessarily increase with increasing P. This suggests that optimal dimensionality selection might be an important consideration.
  • Static Supervised Adaptation: The experiments were conducted in a static, supervised manner, meaning the adaptation data was perfectly transcribed. Real-world applications often involve unsupervised adaptation, which is a more challenging scenario.
  • Focus on Mean Vectors: The adaptation primarily focused on the mean vectors of the HMM Gaussian mixture components. Adapting other parameters (like variances or mixture weights) could potentially yield further improvements.

The authors suggest that the effectiveness of their PPCA-based approach, especially with limited data, makes it a promising direction for future research in speaker adaptation.

Summary updated

Abstract

In this paper, we propose a Bayesian speaker adaptation technique based on the probabilistic principal component analysis (PPCA). The PPCA is employed to obtain the canonical speaker models which provide the a priori knowledge of the training speakers. The proposed approach is conveniently incorporated into the Bayesian adaptation framework where the parameters are adapted to the new speaker’s speech according to the maximum a posteriori (MAP) criterion. Through a number of continuous digit recognition experiments, we can find the effectiveness of the PPCA-based approach compared to the other adaptation approaches with a small amount of adaptation data.
BibTeX
@inproceedings{kim2000bayesian,
  author = {Dong Kook Kim and Nam Soo Kim},
  title = {{Bayesian speaker adaptation based on probabilistic principal component analysis}},
  booktitle = {Interspeech},
  year = {2000},
  url = {https://openreview.net/forum?id=EaJaf5CJha},
}
Figure