TL;DR
This paper revisits and enhances the Time Conditioned Search (TCS) algorithm for large vocabulary continuous speech recognition (LVCSR). By introducing several novel techniques, the authors demonstrate that TCS can now outperform the traditional Word Conditioned Search (WCS), especially when dealing with complex acoustic models and large language models.
What Problem Does This Paper Address?
Automatic Speech Recognition (ASR) systems need to efficiently search through a vast space of possible word sequences to find the most likely transcription of spoken audio. Two common search strategies are Word Conditioned Search (WCS) and Time Conditioned Search (TCS). While WCS has been a standard, TCS theoretically offers advantages for large vocabularies and complex language models, but has historically struggled to match WCS performance, particularly when incorporating advanced acoustic modeling techniques like "across word modelling" (which accounts for how sounds blend at word boundaries). This paper aims to bridge this gap by re-evaluating TCS and proposing new methods to make it competitive, and even superior, to WCS.
Methods
The authors propose several novel extensions to the Time Conditioned Search (TCS) algorithm:
- Early Word End Pruning: This technique addresses the large number of potential word endings that TCS generates. It involves two steps: acoustic pruning applied early in the process and an "anticipated LM pruning" that uses probabilities from already processed hypotheses to prune further ones.
- Context Recombination (CRi): This method aims to reduce redundancy by recombining word end hypotheses that share the same word history but have different start times, if their probabilities are sufficiently similar. This is performed at specified intervals to avoid excessive computational cost.
- Startup Interval: Instead of starting a new search tree at every single time frame (as in standard TCS), this technique allows for skipping certain time frames. This reduces the number of trees initiated, especially when the acoustic model can compensate for the skipped frames.
- Integration with Across Word Modelling: A key contribution is a method to combine TCS with "across word modelling." This is achieved by partitioning the complex search graph of across-word models into a coarticulated root network (handled by WCS) and multiple time-conditioned subtrees (handled by TCS).
Datasets
- NAB'94 H1 development corpus: Used for experiments with within-word modelling, with vocabularies of 20k and 60k words, and 3-gram language models.
- EPPS English 2006 evaluation corpus (from the TC-STAR project): Used for experiments with across-word modelling, featuring a 60k word vocabulary, a 4-gram language model (perplexity 129), and 192 minutes of speech data from European Parliament sessions.
Key Insights and Ideas
- TCS can outperform WCS: With the proposed enhancements, TCS can achieve better performance (lower Word Error Rate) than WCS, even when WCS benefits from advanced techniques like bigram LM look-ahead.
- Efficiency gains from pruning: Early Word End Pruning significantly reduces the computational effort associated with word end expansion, making TCS more efficient and competitive in terms of runtime.
- Context Recombination offers speedups: By intelligently merging similar hypotheses, Context Recombination can reduce the search space and improve runtime without a significant impact on accuracy.
- Startup Interval balances efficiency and accuracy: Skipping time frames during tree initiation can lead to substantial speedups, but choosing the right interval is crucial to avoid performance degradation.
- TCS's independence from vocabulary/LM size: The structure of TCS's search space is less dependent on vocabulary and language model size, which is a significant advantage for very large and complex systems.
Key Results
- Early Word End Pruning: Reduced the runtime effort for word end expansion from ~20% to ~5% on the NAB'94 H1 dev corpus.
- Startup Interval: Increasing the startup interval from 1 to 2 on the EPPS corpus improved performance, achieving a reduction in Real Time Factor (RTF) by a factor of 1.3 for a given Word Error Rate (WER).
- Context Recombination: For a WER of 13.1% on the EPPS corpus, context recombination reduced the RTF by 7%.
- Combined TCS (Interval 2 + CR5): This configuration significantly outperformed WCS (with or without bigram look-ahead) on the EPPS corpus, achieving a better WER and RTF trade-off. For instance, at a WER of 13.1%, this combined TCS achieved a better RTF than WCS with unigram look-ahead and was comparable to WCS with bigram look-ahead, while offering a better overall trade-off.
Limitations and Future Work
- Hybrid Search for Across Word Modelling: The current integration of TCS with across word modelling results in a hybrid approach where the coarticulated root network is still handled as a word-conditioned tree. Future work aims to make the entire search space time-conditioned.
- Context-Dependent LM Look-ahead: Integrating context-dependent (e.g., bigram) LM look-ahead into TCS, which is highly beneficial in WCS, requires further investigation.
- Startup Interval Formulation: The startup interval could potentially be formulated more elegantly as a TCS-specific pruning problem.
Summary updated