TL;DR
This paper presents an efficient hardware accelerator for Chinese speech recognition on edge devices. By binarizing convolutional neural network (CNN) weights and employing a novel Multichannel Shared Computation (MCSC) architecture, they significantly reduce hardware resource consumption and power usage while maintaining acceptable accuracy. This work paves the way for more capable AI on resource-constrained embedded systems.
What Problem Does This Paper Address?
Real-time, offline speech recognition on edge AI devices faces a significant challenge: the high computational and memory demands of modern neural networks. While GPUs are powerful, their power consumption is prohibitive for many edge applications. Field-Programmable Gate Arrays (FPGAs) offer a more power-efficient alternative, but designing efficient CNN accelerators for them is complex. Existing solutions often struggle with balancing accuracy, computational load, and hardware resource usage, especially for complex tasks like speech recognition with variable input lengths. This paper specifically targets the need for a lightweight, energy-efficient accelerator for Chinese speech recognition that can operate effectively on edge devices.
Methods
The core of this work lies in a combination of model optimization and custom hardware design:
- Binary Weighting: Convolutional layer weights are binarized (quantized to +1 or -1). This drastically reduces the number of parameters and simplifies computations, replacing multiplications with additions or subtractions.
- Lightweight CNN Model: A VGG-16 architecture is adapted for speech recognition, with its fully-connected layers reduced and convolutional layers increased. This model is trained using transfer learning with binarized weights.
- Multichannel Shared Computation (MCSC) Architecture: This novel architecture maximizes the reuse of weight and feature map data. It allows for parallel processing of multiple network layers within a single convolutional computing engine.
- Binary Processing Engine (PE): A specialized PE is designed for binary weight convolution, specifically avoiding the need for Digital Signal Processors (DSPs), thus saving significant hardware resources. It uses only seven Look-Up Tables (LUTs) for binary multiplication.
- Ping-Pong Storage: This technique is used for input feature maps to minimize idle time and improve throughput by allowing simultaneous data storage and computation.
- Custom Instruction Set: A configurable instruction set is developed to adapt the accelerator to variable-length voice inputs and different network structures, enabling a reconfigurable design.
Datasets
The primary dataset used for training and evaluation is the ST CMD (Chinese Standard Mandarin Corpus). This dataset is described as containing over 100,000 speech files and approximately 100 hours of speech data, collected from online chats and intelligent speech control sentences, with a balanced representation of male and female voices. The paper does not mention introducing any new benchmark datasets.
Key Insights and Ideas
- Binarization is Key for Edge Efficiency: The paper strongly demonstrates that binarizing weights is a highly effective strategy for reducing hardware resource consumption (especially DSPs and LUTs) and power usage in CNN accelerators, making them suitable for edge devices.
- MCSC Architecture for Data Reuse: The proposed MCSC architecture is a significant contribution, enabling efficient data sharing and parallel processing across multiple MAC units, which is crucial for maximizing throughput in CNNs.
- DSP-Free Binary PE: Designing a binary convolution PE that doesn't require a DSP is a major win for resource-constrained FPGAs, showcasing how clever design can bypass expensive hardware blocks.
- Configurability for Flexibility: The custom instruction set is vital for adapting the accelerator to the inherent variability of speech data (different lengths) and potentially different network configurations, making the design more versatile.
- Accuracy vs. Efficiency Trade-off is Manageable: The results show that binarization, while impacting accuracy, does so within acceptable limits for speech recognition, especially when combined with careful model design and training.
Key Results
- Processing Time:
- 2.24 seconds of speech processed in 69.8 ms.
- 8 seconds of speech processed in 189.51 ms.
- Performance:
- Convolution performance reached 35.66 GOPS/W (for the convolution layer, using 0 DSPs).
- Overall accelerator performance achieved 29.36 GOPS/W.
- Resource Savings: The binary convolution PE uses only 7 LUTs compared to standard 8-bit multiplication which uses 72 LUTs or 1 DSP.
- Power Consumption: The entire accelerator system consumed 2.41 W at 150 MHz, with the core hardware accelerator logic consuming meager resources.
- Speedup: Compared to an 8-bit weighted network, the 1-bit binary network achieved a processing speed improvement of 20.57%. When compared to a GPU, the hardware-accelerated binary network showed a 22.08 times speed improvement.
Limitations and Future Work
- Accuracy: While the paper claims acceptable error rates, the exact PER is not explicitly stated for the final optimized model in the abstract, though Table 2 and 7 suggest it's in the range of 16-18%. Further improvements in accuracy with binarization, especially for more complex tasks, could be explored.
- Fully-Connected Layers: The paper mentions that the last fully-connected layer is quantized to 8 bits. Future work aims to explore the possibility of binarizing fully-connected layers with guaranteed accuracy to achieve even more lightweight models.
- Generalizability: While the custom instruction set offers configurability, further research into broader applicability to diverse network architectures and tasks beyond speech recognition could be beneficial.
- Dataset Size: While ST CMD is a substantial dataset, exploring performance on even larger and more diverse speech datasets could provide a more comprehensive evaluation.
Summary updated