A One-Pass Real-Time Decoder Using Memory-Efficient State Network

Jian Shao, Ta Li, Qingqing Zhang, Qingwei Zhao, Yonghong Yan

Abstract

This paper presents a highly efficient one-pass decoder for Large Vocabulary Continuous Speech Recognition (LVCSR). By integrating static network optimization with dynamic language model application, the proposed system achieves significant speedups and memory reductions compared to standard toolkits like HTK. The architecture utilizes a multi-layer state network and layer-dependent pruning to maintain real-time performance with minimal accuracy loss.

1. Introduction

Large Vocabulary Continuous Speech Recognition (LVCSR) requires searching through vast state spaces, a task that is computationally intensive. Traditional approaches often struggle with the trade-off between the speed of static expansion (e.g., Weighted Finite State Transducers) and the memory efficiency of dynamic expansion. This work introduces a hybrid decoding scheme that statically optimizes lexicon and acoustic models into a multi-layer state network while dynamically applying language models.

2. Problem Statement

The primary challenge in LVCSR decoders is managing the "memory explosion" associated with pre-compiling complex language models into static networks, or the high computational cost of building search networks on the fly. There is a critical need for a decoder that maintains real-time performance on consumer-grade hardware while handling large-scale spontaneous speech tasks without excessive resource consumption.

3. Methodology

The proposed system, TDecode, utilizes several key innovations to optimize the search process:

3.1 Multi-Layer State Network

The search space is organized into a multi-layer state network that integrates lexicons, phonetic contexts (triphones), and HMM states. As shown in Figure 1, the network is divided into Initial, Mid-part, and Final sections. Dummy nodes are employed to cluster fan-in and fan-out triphones, reducing redundancy and improving search efficiency.

The multi-layer state network architecture, illustrating the Initial, Mid-part, and Final sections used for efficient decoding.

3.2 State-Level Optimization and the WI Layer

Optimization is performed at the individual HMM state level using a recursive forward and backward node-merge process. This drastically shrinks the network footprint—reducing memory for within-word networks by a factor of 4 and cross-word networks by a factor of 16. A unique "Word Identity" (WI) layer is placed in the non-shared middle part of the network, creating a bottleneck structure. This allows the decoder to identify specific words early in the search process, facilitating more aggressive pruning.

4. Experimental Setup

The system was evaluated on a Mandarin spontaneous speech task using approximately 200 hours of training data, including CallHome, CallFriend, and "863" datasets. Testing was conducted on the 863Test05 benchmark and the HDev04-II development set using a trigram language model and cross-word triphone acoustic models.

5. Results and Discussion

TDecode was compared against the HDecode tool from the industry-standard HTK toolkit. Key findings include:

  • Speed: TDecode operated five times faster than HDecode.
  • Memory: The memory footprint was reduced by approximately 50% (385 MB vs. 752 MB for trigram tasks).
  • Accuracy: The Character Error Rate (CER) remained within 1% of the baseline, occasionally outperforming it due to refined silence handling.

The effectiveness of the layer-dependent pruning strategy is illustrated in Figure 6. By applying tighter beams at the WI layer and beyond, where word identity is more certain, the search space is significantly reduced without a substantial loss in accuracy.

Character Error Rate (CER) vs. average number of tokens for different pruning layers, demonstrating the efficiency of layer-dependent pruning.

6. Conclusion

The hybrid decoding architecture presented here offers a robust solution for real-time LVCSR. By leveraging state-level merging and a bottleneck-structured WI layer, the system achieves a superior balance of speed and memory efficiency. Future work will explore the trade-offs in lattice quality and the adaptation of this architecture to languages with higher morphological variety.

Summary updated

BibTeX
@article{shao2008one,
  author = {Jian Shao and Ta Li and Qingqing Zhang and Qingwei Zhao and Yonghong Yan},
  title = {{A One-Pass Real-Time Decoder Using Memory-Efficient State Network}},
  journal = {IEICE Trans. Information and Systems},
  year = {2008},
  doi = {10.1093/ietisy/e91-d.3.529},
}
Figure