TL;DR
This paper introduces EKOS, a novel ensemble-based system for robust keyword spotting (KWS) in voice assistants. EKOS defends against both accidental (privacy-leaking) and adversarial (security-threatening) misactivations by leveraging spatial diversity from the acoustic environment and the inherent redundancy in speech signals, all while preserving natural accuracy.
What Problem Does This Paper Address?
Voice assistants (VAs) rely on Keyword Spotting (KWS) to detect wake words like "Alexa" or "Hey Google." This process typically involves an on-device model to quickly identify the keyword, followed by a more powerful cloud-based model for verification and command processing. However, this two-stage system faces significant privacy and security challenges:
- Accidental Activations: Imperfect on-device KWS models can misinterpret non-keyword sounds, leading to unintended activations. This can result in private conversations being recorded and uploaded to the cloud, posing a serious privacy risk. Previous studies found leaks of up to 10 seconds, but this work shows it can be up to a minute.
- Adversarial Activations: Malicious actors can craft "adversarial examples" – subtle audio modifications – to trick the KWS system into activating the VA. This allows them to potentially control connected devices, access sensitive information, or disrupt services, compromising device integrity and user security.
Existing defenses often rely on generic machine learning techniques that can degrade natural accuracy or fail to provide robust protection against sophisticated attacks.
Methods
EKOS tackles these problems with a two-pronged approach:
- Ensemble Learning for Accidental Activations: EKOS leverages the diversity of the acoustic environment and multiple devices. It uses an ensemble of KWS models, each potentially running on a different device (e.g., smartphone, laptop) with varying microphones and acoustic channels. By combining the predictions of these diverse models, EKOS aims to decorrelate their errors, thereby reducing the false positive rate responsible for accidental activations.
- Feature Slicing for Adversarial Activations: To combat adversarial attacks, EKOS exploits the inherent redundancy in speech signals. It decomposes the audio spectrogram into multiple frequency "slices" (bands). Each slice contains partial information about the speech content. EKOS then trains different KWS models on these slices, often using diverse architectures. This "feature slicing" makes it significantly harder for an adversary to craft a single perturbation that fools all models simultaneously, as an attack optimized for one slice might not transfer to others.
- Randomized Ensemble: At inference time, EKOS further enhances robustness by randomly selecting which feature slices and which model architectures to use for each device in the ensemble. This randomization forces adversaries to account for a much larger attack surface.
Datasets
- Speech Commands Dataset [63]: Used for training and testing open-source KWS models. It contains approximately 65,000 one-second utterances of 30 short words.
- Amazon Echo's Wake Keywords Dataset: A new dataset generated by the authors for evaluating EKOS on commercial VAs, including keywords like {Alexa, Computer, Amazon, Echo}.
- Accidental Activation Dataset: Derived from previous work [23, 53], this dataset includes n-gram English phrases phonetically similar to keywords, synthesized to trigger misactivations.
- Devil's Whisper 2 Adversarial Examples: Used for testing against commercial VAs, this is an existing adversarial attack dataset.
Key Insights and Ideas
- Privacy Leakage is Worse Than Expected: Accidental activations can lead to significantly longer private speech recordings (up to a minute) being uploaded to the cloud than previously reported.
- Speech Redundancy is Key: The inherent harmonic structure of speech provides a powerful defense mechanism. By splitting speech into frequency bands and training separate models, EKOS makes it much harder for adversaries to craft universal attacks.
- Ensembling Across Devices and Features: Combining predictions from multiple devices (spatial diversity) and multiple frequency slices (feature diversity) is crucial for both reducing accidental activations and increasing the cost of adversarial attacks.
- Randomization is a Powerful Defense: Randomizing slice-architecture combinations at runtime forces adversaries to consider a much larger attack space, significantly increasing their effort.
- Trade-offs are Manageable: EKOS demonstrates that it's possible to improve robustness against both accidental and adversarial activations without significantly sacrificing natural accuracy or introducing prohibitive latency.
Key Results
- Reduced Accidental Activations: EKOS's ensemble approach significantly reduces false activations, improving privacy. For example, in simulated environments, ensembles of size 3-5 outperformed individual devices.
- Increased Adversarial Attack Cost: EKOS demonstrably increases the cost for adversaries to launch successful attacks. In simulated environments, EKOS required higher perturbation budgets to achieve the same false activation rates compared to baseline models. Over-the-air experiments also showed significantly reduced adversarial success rates for EKOS ensembles.
- Preserved Natural Accuracy: EKOS with feature slicing maintained natural accuracy comparable to baseline models, especially with larger ensemble sizes (l ≥ 5).
- Low Latency Overhead: The system's latency was found to be comparable to existing cloud KWS verification processes, ensuring a good user experience.
- Commercial VA Performance: EKOS demonstrated improved accuracy and reduced false activations on commercial Amazon Echo devices, even without full end-to-end integration of its feature slicing component.
Limitations and Future Work
- Distance Bounding: Devices far from the user might have lower Signal-to-Noise Ratios (SNR), potentially increasing false negatives. The authors suggest incorporating distance bounding mechanisms.
- Integration with Other Robustness Measures: EKOS can be combined with techniques like adversarial training, data augmentation, and voiceprint-based user verification for even greater robustness.
- Physical Channel Approximation: While EKOS accounts for acoustic channels, the simulations are approximations. Real-world physical transformations (e.g., microphone non-linearity) can still pose challenges.
- Computational Overhead: While managed, EKOS does introduce some computational overhead on commodity devices, especially when running multiple models.
- Adaptive Attacks: While EKOS increases the cost, the paper acknowledges that highly sophisticated adaptive attacks could still be challenging.
The authors envision EKOS as a flexible system that can be tuned for different utility-robustness trade-offs by adjusting the number of devices, architectures, and feature slicing configurations.
Summary updated