Tree-based estimation of speaker characteristics for speech recognition

Mats Blomberg, Daniel EleniusInterspeech 2009

TL;DR

This paper introduces a novel tree-based search strategy for efficiently estimating speaker characteristics (like vocal tract length) to adapt speech recognition models. By organizing speaker profiles in a hierarchical tree, they achieve significant computational speedups (up to 99.6%) compared to exhaustive searches, while maintaining or even improving recognition accuracy, especially when adapting adult models to child speech.

What Problem Does This Paper Address?

Speaker adaptation aims to improve speech recognition by adjusting acoustic models to match a specific speaker's voice. A promising approach is to directly modify model parameters based on speaker characteristics like vocal tract length (VTLN). However, a major hurdle is the computational cost of finding the optimal speaker characteristics. Exhaustive grid searches over possible characteristic values become prohibitively slow, especially when considering multiple characteristics simultaneously. Existing methods to reduce this cost often sacrifice recognition accuracy. This paper tackles the challenge of making speaker adaptation computationally efficient without compromising performance, particularly in scenarios like adapting adult models to child speech, where significant acoustic differences exist.

Methods

The core innovation is a tree-based representation of speaker profiles. Instead of searching a continuous space of speaker characteristics, the authors discretize the space and organize it into a multi-dimensional tree.

  • Tree Generation: The process starts with a top-down split of the speaker profile domain. Each node in the tree represents a range of speaker characteristics. Leaf nodes correspond to specific, quantized speaker profile vectors. Acoustic model sets are generated for each leaf node by transforming a conventionally trained model set using the leaf's profile vector. Non-leaf nodes store model sets created by merging the models of their child nodes using an efficient, computationally inexpensive algorithm based on second moments.
  • Efficient Merging: The paper proposes a novel, computationally efficient method for merging Gaussian mixture components from child nodes into parent nodes. This method leverages the fact that child node models are derived from the same original model and have similar transformation features, avoiding costly cost function computations.
  • Tree Search: During recognition, a maximum-likelihood criterion guides a search through the tree. Instead of evaluating every possible combination (like an exhaustive search), the recognizer traverses the tree, selecting the most promising child node at each level. This significantly reduces the number of model sets that need to be evaluated. The search can stop at any level, allowing for a trade-off between computational load and accuracy.

Datasets

  • PF-Star-Sw: A Swedish children's speech corpus containing connected digits from children aged 4-8 years. This corpus was downsampled and re-quantized for the experiments.
  • TIDIGITS: A widely used speaker-independent connected digits corpus.
  • SPEECON database (adult Swedish part): Used for training adult models.

The paper highlights the challenge of adapting adult models to child speech due to age-related differences.

Key Insights and Ideas

  • Hierarchical structure unlocks efficiency: Organizing speaker characteristics in a tree allows for a drastically reduced search space, making complex multi-dimensional adaptation feasible.
  • Root node is surprisingly effective: Even the model set at the root of the tree (representing a broad range of speaker characteristics without specific estimation) provides substantial improvement over baseline models. This suggests that even a general adaptation can be beneficial.
  • Efficient merging is key: The proposed second-moment-based merging algorithm is significantly faster than traditional greedy methods without sacrificing accuracy, making the tree construction practical.
  • Trade-off between depth and accuracy: Stopping the tree search at intermediate levels (e.g., two levels below the root) offers a good balance between computational load and recognition performance. Going too deep can lead to overly specialized models that struggle with non-stationarity.
  • Multi-dimensional adaptation pays off: Using four speaker characteristics (VTLN, spectral slope, model variance scaling) consistently improves performance over single-dimensional adaptation, demonstrating the benefit of jointly optimizing multiple speaker properties.

Key Results

  • Significant Speedup: The tree-based search reduces computational load to a fraction of an exhaustive grid search. For the 4-dimensional case, the computational load at the optimum stop-level is around 0.4% of that of an exhaustive search.
  • Comparable Accuracy: The tree-based approach achieves recognition accuracy comparable to or better than exhaustive grid search.
  • Best Recognition Results: The paper reports Word Error Rates (WER) as low as 0.93% for PF-Star-Sw (adult models) and 1.00% for TIDIGITS (adult models), using the 1-D VTLN adaptation. With 4-D adaptation, the best WERs are 1.02% for PF-Star-Sw and 0.93% for TIDIGITS.
  • Efficient Merging Performance: The proposed second moment merging algorithm is much faster (e.g., 29s vs. 439s for PF-Star-Sw GMM-16) than the greedy algorithm, with no visible performance degradation in recognition error rate.

Limitations and Future Work

  • Quantization Resolution: The paper notes that while deeper levels in the tree offer higher quantization resolution, this can be counteracted by increasingly narrow models that may struggle with non-stationary speaker profiles.
  • Spontaneous vs. Read Speech: The current set of speaker characteristics might not fully capture differences between spontaneous and read speech.
  • Phone-Specific Factors: Ongoing work aims to incorporate phone-specific factors, which could further benefit from the reduced search space provided by the tree-based approach.
  • Further Tree Size Reduction: The authors suggest that merging acoustically similar leaf nodes, even if non-adjacent, could further reduce tree size and improve efficiency, potentially using standard clustering techniques.
  • Expanding Speaker Characteristics: Future work will focus on expanding the set of speaker characteristics considered, including spontaneous/read speech dimensions, speech rate, and regional accent.

Summary updated

BibTeX
@inproceedings{blomberg2009tree,
  author = {Mats Blomberg and Daniel Elenius},
  title = {{Tree-based estimation of speaker characteristics for speech recognition}},
  booktitle = {Interspeech},
  year = {2009},
}
Figure