Vau Da Muntanialas: Energy-Efficient Multi-Die Scalable Acceleration of RNN Inference
Lists
Only you see this
Abstract
Recurrent neural networks such as Long Short-Term Memories (LSTMs) learn temporal dependencies by keeping an internal state, making them ideal for time-series problems such as speech recognition. However, the output-to-input feedback creates distinctive memory bandwidth and scalability challenges in designing accelerators for RNNs. We present Muntaniala, an RNN accelerator architecture for LSTM inference with a silicon-measured energy-efficiency of 3.25TOP/s/W and performance of 30.53GOP/s in UMC 65nm technology. The scalable design of Muntaniala allows running large RNN models by combining multiple tiles in a systolic array. We keep all parameters stationary on every die in the array, drastically reducing the I/O communication to only loading new features and sharing partial results with other dies. For quantifying the overall system power, including I/O power, we built Vau da Muntanialas, to the best of our knowledge, the first demonstration of a systolic multi-chip-on-PCB array of RNN accelerator. Our multi-die prototype performs LSTM inference with 192 hidden states in 330 <inline-formula> <tex-math notation="LaTeX">$\mu \text{s}$ </tex-math></inline-formula> with a total system power of 9.0 mW at 10 MHz consuming 2.95 <inline-formula> <tex-math notation="LaTeX">$\mu \text{J}$ </tex-math></inline-formula>. Targeting the 8/16-bit quantization implemented in Muntaniala, we show a phoneme error rate (PER) drop of approximately 3% with respect to floating-point (FP) on a 3L-384NH-123NI LSTM network on the TIMIT dataset.
BibTeX
@article{paulin2022vau,
author = {Gianna Paulin and Francesco Conti and Lukas Cavigelli and Luca Benini},
title = {{Vau Da Muntanialas: Energy-Efficient Multi-Die Scalable Acceleration of RNN Inference}},
journal = {IEEE Trans. Circuits Syst. I Regul. Pap.},
year = {2022},
doi = {10.1109/tcsi.2021.3099716},
}| Chipmunk: A systolically scalable 0.9 mm2, 3.08Gop/s/mW @ 1.2 mW accelerator for near-sensor recurrent neural network inference | arXiv · 2017 |
| Efficient Hardware Architectures for 1D- and MD-LSTM Networks | Journal of Signal Processing Systems · 2020 |
| SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks | arXiv · 2019 |
| Hyperdrive: A Systolically Scalable Binary-Weight CNN Inference Engine for mW IoT End-Nodes | arXiv · 2018 |
| EdgeDRNN: Recurrent Neural Network Accelerator for Edge Inference | IEEE J. Emerg. Sel. Topics Circuits Syst · 2020 |
| Hybrid Systolic Array Accelerator with Optimized Dataflow for Edge Large Language Model Inference | arXiv · 2025 |