Abstract
Voice-triggered smart assistants often rely on detection of a trigger-phrase before they start listening for the user request. Mitigation of false triggers is an important aspect of building a privacy-centric non-intrusive smart assistant. In this paper, we address the task of false trigger mitigation (FTM) using a novel approach based on analyzing automatic speech recognition (ASR) lattices using graph neural networks (GNN). The proposed approach uses the fact that decoding lattice of a falsely triggered audio exhibits uncertainties in terms of many alternative paths and unexpected words on the lattice arcs as compared to the lattice of a correctly triggered audio. A pure trigger-phrase detector model doesn't fully utilize the intent of the user speech whereas by using the complete decoding lattice of user audio, we can effectively mitigate speech not intended for the smart assistant. We deploy two variants of GNNs in this paper based on 1) graph convolution layers and 2) self-attention mechanism respectively. Our experiments demonstrate that GNNs are highly accurate in FTM task by mitigating ~87% of false triggers at 99% true positive rate (TPR). Furthermore, the proposed models are fast to train and efficient in parameter requirements.
Introduction
Smart assistants like Siri or Alexa rely on "voice triggering"—listening for a specific wake-word. However, these systems often suffer from False Triggers (FT), where background noise or phrases that sound similar to the wake-word (e.g., "Hey Don" instead of "Hey Siri") accidentally wake the device. This is a major hurdle for both user experience and privacy.
The challenge is that a simple trigger-phrase detector only looks at the audio snippet of the wake-word itself, ignoring the context of the speech that follows. Previous attempts to solve this used the "1-best" hypothesis, but this often misses the uncertainty inherent in the speech recognition process. This paper addresses the need for a more robust, efficient way to distinguish between intended commands and accidental noise by looking at the entire "lattice" of possibilities the recognizer considered.
Methodology
The authors propose using Graph Neural Networks (GNNs) to process Automatic Speech Recognition (ASR) lattices. An ASR lattice is essentially a Directed Acyclic Graph (DAG) where nodes are time-stamps and arcs are hypothesized words.
Each arc in the graph is represented by a 20-dimensional vector including acoustic model scores, language model scores, duration, and "bag-of-phones" embeddings. The paper explores three main GNN architectures:
- Graph Convolutional Networks (GCN): These aggregate information from immediate neighboring arcs. To make these networks deeper, the authors used Residual Blocks (ResGCN).
- Self-Attention GNN (SAGNN): Based on the Transformer architecture, this allows any arc in the lattice to "attend" to any other arc.
- Masked SAGNN: This sophisticated variant combines self-attention with the structural constraints of the lattice. It uses the lattice's adjacency matrix to "mask" the attention mechanism, ensuring that arcs only aggregate information from their actual neighbors in the graph.
Key Insights: Lattices as Structural Evidence
A "true" trigger usually results in a "skinny" lattice with one clear, high-confidence path. A "false" trigger (like background noise) results in a "messy" lattice with many competing, low-confidence paths. GNNs are uniquely suited to "see" this structural difference.
By masking the attention with the lattice structure, the model focuses on linguistically and temporally valid connections. Furthermore, GNNs allow for much more efficient parallelization than traditional Lattice-RNNs, leading to significantly faster training times.
Experimental Results
The study used an internal Apple dataset of True Triggers (~42k training) and False Triggers (~18k training), augmented 3x for robustness. The evaluation focused on a high True Positive Rate (TPR > 0.99) regime.
- Accuracy: The MaskedSAGNN achieved a False Alarm Rate (FAR) of 0.134 at a 99% TPR, effectively mitigating ~87% of false triggers.
- Speed: GNN-based models were significantly faster to train. While a Bi-directional Lattice RNN took ~8 minutes per epoch, the GNN models took only ~1.5 minutes per epoch—a 5.3x speedup.
- Depth: Using Residual GCNs allowed the researchers to go deeper (up to 16 layers) while maintaining high performance, whereas vanilla GCNs peaked at 6 layers.
Conclusion
Lattice-based GNNs offer a powerful and efficient approach to false trigger mitigation. By leveraging the structural uncertainty of ASR lattices, these models significantly outperform simple trigger-phrase detectors. Future work could involve end-to-end learning from raw acoustic features and applying this lattice-based approach to tasks like user-intent classification.
Summary updated