Initial Experiments with Estonian Speech Recognition.

Anton RagniNODALIDA 2007 · OpenReview

Abstract

This paper establishes a baseline for Estonian automatic speech recognition (ASR) using a traditional word-based approach. By leveraging standard Hidden Markov Model (HMM) techniques and the HTK toolkit, the study achieves a Word Error Rate (WER) of 37.5% on the Estonian portion of the Babel database. These results provide a benchmark for future research into this complex, agglutinative language, highlighting the impact of acoustic model complexity and hyperparameter tuning.

Introduction

Estonian is an inflectional and agglutinative language, posing significant challenges for speech recognition. A single root word can generate a vast number of derivative forms through prefixes, suffixes, and compounding, making a comprehensive word-based vocabulary difficult to maintain. While many researchers utilize subword units (e.g., morphemes) to address this, this study investigates the performance of a standard word-based system to provide an independent baseline.

Methodology

The recognition pipeline was constructed using the HMM Tool Kit (HTK). Mel-Frequency Cepstral Coefficients (MFCC) were used for feature extraction, and the acoustic units were based on an orthographic set of 29 monophones.

Language models (LM) were trained on the Mixed Corpus of Estonian (MCE), containing approximately 77 million words. The vocabulary was limited to the 65,000 most frequent words. The study compared bigram and trigram models, experimenting with various n-gram "cut-off" values to manage model size and complexity. Decoding was performed using the HDecode one-pass recognizer.

Experimental Setup

The acoustic models were trained and evaluated using the Estonian part of the Babel Multi-Language Speech Database. The training set comprised 7.4 hours of audio from 163 passages. The development and test sets were smaller, consisting of 0.3 and 0.9 hours respectively.

Results and Discussion

The experiments focused on fine-tuning several decoding hyperparameters, including the word insertion penalty, beam width, and the number of Gaussian mixture components (pdfs) per state.

Hyperparameter Tuning

The word insertion penalty proved to be a critical factor in balancing the trade-off between insertion and deletion errors. As shown in Figure 1, the WER follows a characteristic U-curve, with optimal performance achieved at a penalty of approximately -40.

WER vs. Word Insertion Penalty

Acoustic Model Complexity

One of the most significant performance improvements resulted from increasing the number of Gaussian probability density functions (pdfs) in the HMM states. Moving from three to five pdfs yielded a 25% relative improvement in WER. Beyond five mixtures, the gains plateaued, suggesting a limit to the complexity that could be supported by the 7.4-hour training set.

WER vs. Number of Gaussian Mixtures

Error Analysis

Substitution errors accounted for a staggering 75% of all errors. This indicates that while the system often detects speech, it struggles to distinguish between similar-sounding words. Furthermore, trigram models offered only marginal improvements over bigrams, with a trigram "hit-ratio" of only 11% on the test set, underscoring the data sparsity issues inherent in highly inflected languages.

Conclusion

The study achieved a lowest WER of 37.5% using a bigram model with no cut-offs and a trigram model with a cut-off of 1. While this error rate is higher than those typically seen in English ASR, it represents a solid baseline for word-based Estonian recognition given the limited training data. Future work will focus on multi-pass decoding strategies and the incorporation of subword units to mitigate the "Out-of-Vocabulary" (OOV) problem caused by Estonian's agglutinative morphology.

Summary updated

Abstract

Anton Ragni. Proceedings of the 16th Nordic Conference of Computational Linguistics (NODALIDA 2007). 2007.
BibTeX
@inproceedings{ragni2007initial,
  author = {Anton Ragni},
  title = {{Initial Experiments with Estonian Speech Recognition.}},
  booktitle = {NODALIDA},
  year = {2007},
  url = {https://openreview.net/forum?id=sFmyx1ZpVRM},
}
Figure