Lattice Generation in Attention-Based Speech Recognition Models

Michal Zapotoczny, Piotr Pietrzak, Adrian Lancucki, Jan ChorowskiInterspeech 2019 · OpenReview

Abstract

Attention-based neural speech recognition models are frequently decoded with beam search, which produces a tree of hypotheses. In many cases, such as when using external language models, numerous decoding hypotheses need to be considered, requiring large beam sizes during decoding. We demonstrate that it is possible to merge certain nodes in a tree of hypotheses, in order to obtain a decoding lattice, which increases the number of decoding hypotheses without increasing the number of candidates that are scored by the neural network. We propose a convolutional architecture, which facilitates comparing states of the model at different time steps. The experiments are carried on the Wall Street Journal dataset, where the lattice decoder obtains lower word error rates with smaller beam sizes, than an otherwise similar architecture with regular beam search.

Introduction

Most modern Automatic Speech Recognition (ASR) systems use beam search to find the most likely transcript. Beam search produces a "tree" of hypotheses: as the model predicts each new character, it keeps the top $N$ best candidates. However, this approach suffers from significant redundancy. Many paths in the tree might eventually lead to the same state, but because standard Recurrent Neural Network (RNN) decoders have "infinite" memory (their hidden state depends on every previous character), two paths are almost never exactly identical. This prevents the system from merging them, leading to a restricted search space unless computationally expensive large beam sizes are used.

Methods

The core innovation of this work is the transition from an RNN-based decoder to a Temporal Convolutional Network (TCN) decoder. Unlike RNNs, a TCN only looks at a fixed-size window of previous characters. This finite receptive field means that if two different hypotheses end with the same suffix, their decoder states become exactly identical.

The authors propose an algorithm that merges these identical states into a lattice (a Directed Acyclic Graph). Two nodes are merged if their TCN states match, their external language model states are the same, and their attention weights are sufficiently similar. This allows the system to compactly represent an exponential number of possible sentences.

A character-level decoding lattice generated by the proposed TCN-based model.

Key Results

The models were evaluated on the Wall Street Journal (WSJ) dataset. The lattice decoder consistently outperformed standard beam search across various metrics:

  • 3x Efficiency Boost: The lattice decoder achieved similar performance to a standard beam search while using a 3x smaller beam size (e.g., a lattice with beam 50 performed as well as a tree with beam 150).
  • Lower Error Rates: For beam sizes between 20 and 150, the lattice approach consistently yielded lower Word Error Rates (WER). On the Eval92 set, the lattice decoder reached a WER of 6.98% compared to 7.16% for the standard beam.
  • Massive Hypothesis Space: With a beam size of 10, the proposed lattice contained an average of over 12,700 valid paths, providing a much richer search space for the same computational cost as a 10-hypothesis tree.
Word Error Rate (WER) as a function of beam size for standard beam search vs. the proposed lattice decoding.

Conclusion

By leveraging the finite memory of TCNs, this research demonstrates that attention-based models can generate efficient decoding lattices. This technique significantly reduces the redundancy of beam search, allowing for much larger effective search spaces and lower error rates without increasing the number of neural network forward passes. This approach is particularly valuable for complex decoding tasks involving external language models or multi-pass processing.

Summary updated

Abstract

Attention-based neural speech recognition models are frequently decoded with beam search, which produces a tree of hypotheses. In many cases, such as when using external language models, numerous decoding hypotheses need to be considered, requiring large beam sizes during decoding. We demonstrate that it is possible to merge certain nodes in a tree of hypotheses, in order to obtain a decoding lattice, which increases the number of decoding hypotheses without increasing the number of candidates that are scored by the neural network. We propose a convolutional architecture, which facilitates comparing states of the model at different pi The experiments are carried on the Wall Street Journal dataset, where the lattice decoder obtains lower word error rates with smaller beam sizes, than an otherwise similar architecture with regular beam search.
BibTeX
@inproceedings{zapotoczny2019lattice,
  author = {Michal Zapotoczny and Piotr Pietrzak and Adrian Lancucki and Jan Chorowski},
  title = {{Lattice Generation in Attention-Based Speech Recognition Models}},
  booktitle = {Interspeech},
  year = {2019},
  url = {https://openreview.net/forum?id=0wzcPcay5u},
}
Figure