TL;DR
This paper introduces a novel method for compressing speech features using low-dimensional bottleneck features (BNFs) that can be processed on power-efficient Digital Signal Processors (DSPs). This allows for significantly reduced memory footprints and power consumption in on-device speech recognition systems, with minimal impact on accuracy.
What Problem Does This Paper Address?
Modern applications increasingly rely on continuous speech recognition (LVCSR) for rich user interaction. However, running these complex models on mobile devices is power-intensive, quickly draining batteries. The challenge lies in optimizing LVCSR for low-power, always-on hardware like DSPs, which have limited memory and processing capabilities. This paper aims to bridge this gap by developing efficient feature extraction methods that can run on DSPs, enabling more of the speech processing pipeline to occur on these low-power chips, thereby reducing the reliance on the main, power-hungry processor.
Methods
The core of the paper's contribution is the development of efficient bottleneck feature (BNF) extractors designed for DSPs. They adapt a state-of-the-art Listen, Attend, Spell (LAS) end-to-end ASR model by splitting its encoder. The initial layers, responsible for feature extraction, are optimized to run on a DSP.
Key techniques include:
- Quantized Mel-Features (QM-features): These are log-mel based features with a 16-bit fixed-point representation, designed to be compact and easily calculable on a DSP. The paper explores reducing the bandwidth of these features by using fewer frequency bins.
- Bottleneck Feature Extraction (BNF): The paper employs a convolutional structure, specifically separable convolutional layers, optimized for DSPs. These layers are quantized to 8 bits, and biases, batch normalization, and ReLU activations are used.
- Architecture Exploration: The authors systematically explore the parameter space of their BNF architecture, varying output dimension size, quantization level, convolutional stride, kernel size, and the number of layers.
- DSP and Main Processor Split: The LAS model's encoder is split, with the DSP handling the initial, low-power feature compression into BNFs, which are then cached. The main processor handles the subsequent, more computationally intensive parts of the inference.
Datasets
- LibriSpeech 100 corpus: Used for training and evaluating the models. The authors note that a baseline model trained on this corpus yields a WER of 21.8%.
- Proprietary corpus (mentioned): The authors reference a state-of-the-art result from Chiu et al. [2] achieved on a "very large, proprietary corpus" which reported a WER of 4.1%.
Key Insights and Ideas
- DSP as a Feature Compressor: The paper demonstrates that DSPs can be effectively utilized not just for simple signal processing but for complex feature extraction and compression, a novel application for LVCSR.
- Bandwidth vs. Accuracy Trade-off: A clear trade-off exists between feature bandwidth and recognition accuracy. However, the authors show that significant bandwidth reduction is possible with only a minor increase in Word Error Rate (WER).
- Optimal Quantization: For on-device deployment, 4-bit quantization for the bottleneck features, combined with 8-12 dimensional BNFs, appears to be optimal for balancing compression and accuracy.
- Time Compression Limits: The paper suggests that time compression beyond a certain point (around 40 ms) leads to a significant performance degradation, indicating a limit to how much temporal information can be discarded without impacting accuracy.
- Privacy Benefits: By performing more processing on-device, the proposed architecture inherently enhances user privacy as sensitive audio data doesn't need to be sent to the cloud.
Key Results
The paper achieves significant reductions in feature footprint with minimal accuracy loss:
- Factor of 10 reduction: The system can reduce the footprint of standard DSP spectral features by a factor of 10 with no loss in WER.
- Factor of 64 reduction: A reduction by a factor of 64 is achieved with only a 5.8% relative increase in WER.
- Comparison to Baseline: A model achieving ~4.8 kbps bandwidth (compared to the standard QM-features at 51.2 kbps) outperforms the baseline QM-features model with a 0.6% increase in WER.
- Overall Compression: Compared to the original floating-point model, the system achieves a 960x feature compression for a 6.6% increase in WER.
- Best Performing Models: The best performing models achieved bandwidths as low as ~0.6 kbps and ~0.4 kbps, though these resulted in higher WERs (around 30% and 36% respectively), indicating the performance breakdown point.
Limitations and Future Work
- Empirical Battery Data: While the paper argues for power savings by reducing main processor wake-ups, it acknowledges that empirical data is necessary to fully understand the impact on battery consumption, especially considering potential increases in computation on the main processor due to reduced time compression.
- In-Situ Testing: Further tuning and characterization are needed when these models are tested in real-world, noisy speech conditions.
- Computational Cost of Attention/Decoder: Some of the highly compressed models might require more computation in the attention/decoder stages on the main processor, which could offset some of the power savings.
- Future Work: The authors suggest that this architecture could be adopted as a standard technique for speech compression in portable applications. Further tuning for in-situ testing and characterization under noisy conditions are also implied future directions.
Summary updated