A low complexity model adaptation approach involving sparse coding over multiple dictionaries

S. Shahnawazuddin, Rohit SinhaInterspeech 2014

TL;DR

This paper introduces a novel, low-complexity approach to adapting speech recognition models for individual users in real-time. By leveraging sparse coding over multiple dictionaries and cluster-specific eigenvoices, the method significantly reduces the number of parameters to be estimated, leading to faster adaptation and improved accuracy, especially in low-data scenarios.

What Problem Does This Paper Address?

On-line speech recognition systems, like those used for spoken queries, need to adapt quickly to individual users with very little adaptation data. Traditional adaptation methods often require a large amount of data to estimate many model parameters, making them impractical for real-time, low-data scenarios. While existing fast adaptation techniques reduce the number of parameters, they can suffer from performance saturation or fail to capture fine-grained acoustic variations. This paper aims to bridge this gap by proposing an adaptation method that is both computationally efficient and effective in capturing individual speech characteristics with minimal data.

Methods

The paper builds upon existing sparse representation (SR) techniques for adaptation. The core contributions are:

  • Orthogonal Matching Pursuit (OMP) for Bases Selection: Instead of the standard Matching Pursuit (MP), the authors use OMP to select bases from a redundant dictionary. A key advantage of OMP is that it avoids selecting the same "atom" (basis vector) multiple times, preventing singularity issues in subsequent calculations and reducing computational overhead compared to methods that need to explicitly remove redundant atoms.
  • Cluster-Specific Eigenvoices: To better capture local acoustic details, the authors propose using eigenvoices derived from acoustically clustered speaker supervectors. This contrasts with conventional eigenvoices, which capture only global variations across all speakers.
  • Low-Complexity Multiple Dictionary Approach: The main innovation is a model adaptation scheme that uses sparse coding over multiple dictionaries. Instead of interpolating selected bases with estimated weights, the sparse coded representations of the test data are directly used (after scaling) as adapted model mean parameters. This drastically reduces the number of parameters to be estimated, making it suitable for on-line applications. The number of dictionaries can be increased (e.g., using cluster-specific eigenvoices) to further improve performance.

Datasets

  • Training Data: WSJCAMO corpus, consisting of 7861 utterances from 92 speakers (approx. 15.5 hours).
  • Test Data: WSJCAMO corpus, consisting of 368 utterances from 20 speakers (approx. 0.6 hours).

Key Insights and Ideas

  • OMP's Advantage: Using OMP for basis selection is computationally simpler and more robust than MP when dealing with redundant dictionaries, as it inherently avoids redundant atom selection.
  • The Power of Local Variation: Cluster-specific eigenvoices are more effective than global eigenvoices because they capture finer acoustic details specific to groups of speakers, leading to better adaptation.
  • Sparse Coding as Direct Adaptation: Directly using scaled sparse coded supervectors as adapted model parameters is a significant simplification. It bypasses the complex weight estimation step required in traditional interpolation methods, drastically reducing computational load.
  • Scalability with Dictionaries: Increasing the number of dictionaries (e.g., by incorporating cluster-specific eigenvoices) allows for a more nuanced representation of speech variations and can lead to improved adaptation performance, without a proportional increase in the number of parameters to be estimated.

Key Results

  • The proposed redundant dictionary approach achieved a relative improvement of 7.5% over the unadapted SI system.
  • The cluster-specific eigenvoice approach showed improvements over conventional eigenvoices.
  • The multiple dictionary based approach achieved a best-case Word Error Rate (WER) of 10.55% with 4 dictionaries and a sparsity of 12, outperforming the SI system by 6.7%.
  • Crucially, the multiple dictionary approach reduces the number of estimated parameters by a factor of 4 compared to previous methods, making it highly suitable for on-line applications.

Limitations and Future Work

  • Limited Training Data for Clustering: The authors note that for the WSJCAMO dataset, splitting the 92 speakers into four clusters resulted in small cluster sizes, limiting the effectiveness of cluster-specific eigenvoices. They suggest that this extension could be more beneficial on a speech database with a larger number of training speakers.
  • Future Work: The paper implies that extending the multiple dictionary approach to more dictionaries, especially with larger training datasets for better clustering, is a promising direction for further performance gains.

Summary updated

BibTeX
@inproceedings{shahnawazuddin2014low,
  author = {S. Shahnawazuddin and Rohit Sinha},
  title = {{A low complexity model adaptation approach involving sparse coding over multiple dictionaries}},
  booktitle = {Interspeech},
  year = {2014},
}
Figure