Abstract
A liquid state machine (LSM) is a biologically plausible model of a cortical microcircuit. It exists of a random, sparse reservoir of recurrently connected spiking neurons with fixed synapses and a trainable readout layer. The LSM exhibits low training complexity and enables backpropagation-free learning in a powerful, yet simple computing paradigm. In this work, the liquid state machine is enhanced by a set of bio-inspired extensions to create the extended liquid state machine (ELSM), which is evaluated on a set of speech data sets. Firstly, we ensure excitatory/inhibitory (E/I) balance to enable the LSM to operate in edge-of-chaos regime. Secondly, spike-frequency adaptation (SFA) is introduced in the LSM to improve the memory capabilities. Lastly, neuronal heterogeneity, by means of a differentiation in time constants, is introduced to extract a richer dynamical LSM response. By including E/I balance, SFA, and neuronal heterogeneity, we show that the ELSM consistently improves upon the LSM while retaining the benefits of the straightforward LSM structure and training procedure. The proposed extensions led up to an 5.2% increase in accuracy while decreasing the number of spikes in the ELSM up to 20.2% on benchmark speech data sets. On some benchmarks, the ELSM can even attain similar performances as the current state-of-the-art in spiking neural networks. Furthermore, we illustrate that the ELSM input-liquid and recurrent synaptic weights can be reduced to 4-bit resolution without any significant loss in classification performance. We thus show that the ELSM is a powerful, biologically plausible and hardware-friendly spiking neural network model that can attain near state-of-the-art accuracy on speech recognition benchmarks for spiking neural networks.
Introduction
Spiking Neural Networks (SNNs) represent a promising frontier for low-power edge AI, leveraging the brain's inherent energy efficiency. However, training SNNs remains a significant challenge. Conventional methods like Backpropagation-Through-Time (BPTT) are computationally intensive and lack biological plausibility. Liquid State Machines (LSMs) offer a compelling alternative by utilizing a random, fixed reservoir of recurrently connected spiking neurons, where only the final readout layer requires training. While LSMs are computationally efficient and hardware-friendly, they have historically underperformed compared to BPTT-trained models on complex temporal tasks. This work introduces the Extended Liquid State Machine (ELSM), which incorporates bio-inspired mechanisms to bridge this performance gap.
The Extended Liquid State Machine (ELSM)
The ELSM builds upon a standard conductance-based Leaky Integrate-and-Fire (LIF) model by integrating three key biological extensions designed to enhance information processing and memory:
- Excitatory/Inhibitory (E/I) Balance: By tuning inhibitory weights to cancel out excitatory signals, the reservoir is maintained in an "edge-of-chaos" regime. This state is mathematically optimal for complex information processing.
- Spike-Frequency Adaptation (SFA): This mechanism simulates neuronal "fatigue," where a neuron's firing threshold increases temporarily after repeated activity. SFA provides a form of short-term memory, enabling the network to integrate temporal information over longer periods.
- Neuronal Heterogeneity: Instead of uniform membrane time constants, neurons are assigned a range of values. This diversity allows the reservoir to capture both fast transients and slow-moving temporal features in the input data.
Experimental Evaluation
The ELSM was evaluated across four benchmark speech datasets: Spiking Heidelberg Digits (SHD), TI-Alpha, Google Speech Commands (GSC), and N-TIDIGITS. The readout layer utilized a simple Logistic Regression classifier trained on excitatory neuron spike counts.
Key Results
- Performance Gains: The ELSM consistently outperformed the baseline LSM, achieving an average accuracy increase of 5.2%. On the SHD dataset, a 16,000-neuron ELSM reached 89.3% accuracy, rivaling state-of-the-art BPTT-trained models (90.4%).
- Energy Efficiency: The model achieved a 20.2% reduction in total spike counts, demonstrating that bio-inspired extensions can improve accuracy while simultaneously lowering power consumption.
- Hardware Compatibility: Synaptic weights in the reservoir can be quantized to 4-bit resolution without significant performance loss, making the ELSM highly suitable for deployment on specialized neuromorphic chips like Intel’s Loihi.
Discussion and Conclusion
The findings highlight that the "edge of chaos" provided by E/I balance is critical for reservoir dynamics. Furthermore, SFA offers a passive memory mechanism that avoids the complexity of training recurrent weights. While the ELSM shows robust performance, future work could explore simpler neuron models or local learning rules like STDP to automate E/I balancing. Despite a performance gap on high-noise datasets like GSC, the ELSM stands as a powerful, biologically plausible, and hardware-efficient model for spiking neural networks in speech recognition.
Summary updated