Introduction
Automatic Speech Recognition (ASR) systems often struggle with "long-tail" vocabulary—rare terms, proper nouns, or technical jargon that appear infrequently in training data. To mitigate this, researchers employ contextual biasing, providing the model with a "biasing list" of relevant terms (e.g., contact names or presentation keywords) to improve recognition accuracy.
A state-of-the-art approach for this is the Tree-Constrained Pointer Generator (TCPGen), which organizes biasing words into a prefix tree (trie). During decoding, the model queries the tree to identify valid next subword units. However, standard TCPGen is "myopic": nodes only contain information about their specific subword, lacking context about the full words they belong to. This limitation makes it difficult for the model to determine precisely when to transition to the biasing list.
Methodology: GNN Tree Encodings for Lookahead
To address this "myopia," this paper introduces GNN Tree Encodings to provide the model with "lookahead" capabilities. Instead of a node representing an isolated subword, a Graph Neural Network (GNN) aggregates information from all descendant nodes—effectively the remainder of the word—and propagates it up to the current node. This allows the model to "see" the future branches of a word at each decoding step.
The authors evaluated several GNN architectures for this task:
- Tree-RNN: A recursive bottom-up encoding approach.
- Graph Convolutional Network (GCN) & GCNII: Spectral methods that pass messages in parallel across the adjacency matrix.
- GraphSAGE: A spatial method using max-pooling to aggregate features from neighbors.
- Hybrid Approaches: Combinations of GCN and GraphSAGE (Additive or Bilinear) were tested to capture both spectral and spatial graph features.
Efficiency is maintained through a parameter-tying scheme, sharing weights across GNN layers to reduce complexity while allowing deep information propagation.
Experimental Setup and Datasets
The system was evaluated on two primary datasets:
- LibriSpeech: Large-scale simulations were conducted using biasing lists containing rare words from transcriptions mixed with 1,000 "distractor" words to mimic real-world noise.
- AMI Meeting Corpus: This "audio-visual" setup tested the system's robustness in dynamic environments. Biasing lists were automatically extracted via OCR from presentation slides used during meetings.
Key Results and Insights
The integration of GNN lookahead significantly improved performance across all metrics:
- Massive Error Reduction: On LibriSpeech, the GNN-enhanced TCPGen achieved a 60% reduction in Rare Word Error Rate (R-WER) compared to a standard ASR system and a 30-40% reduction over the original TCPGen.
- Lookahead Efficacy: By encoding the "future" of a word, the pointer generator can trigger more decisively at the first syllable of a rare word. This is particularly effective for words consisting of 3 to 4 subword pieces.
- Architectural Synergy: GCNs excel at near-future information, while GraphSAGE preserves salient features from deeper in the tree. Their combination yielded the best overall results.
- Real-World Application: In the AMI experiments, the system achieved a 30-35% relative R-WER reduction using OCR-derived lists, demonstrating its utility for automatically generated context.
Limitations and Future Directions
Despite these gains, performance begins to degrade as biasing lists scale toward 5,000 words due to the quadratic growth of lookahead nodes. Additionally, constructing the adjacency matrix for unique lists introduces minor latency. Future research will explore more complex graph structures beyond prefix trees and optimization techniques for extremely large-scale biasing (tens of thousands of words).
Summary updated