Robust Sound Recognition: A Neuromorphic Approach

Jibin Wu, Zihan Pan, Malu Zhang, Rohan Kumar Das, Yansong Chua, Haizhou Li

Abstract

This paper presents a neuromorphic sound recognition system designed to emulate the human auditory pathway, from the cochlea to the brain's neural networks. By integrating an event-driven front-end with Spiking Neural Networks (SNNs), the architecture achieves energy efficiency several orders of magnitude higher than traditional deep learning models. The system utilizes bio-inspired mechanisms such as auditory masking and neural threshold coding to process sound as asynchronous spikes. Experimental results demonstrate high accuracy in speech and environmental sound classification, even in noise-corrupted environments, making it a prime candidate for low-power, always-on wearable devices.

Introduction

While modern deep learning has achieved human-level accuracy in speech and sound recognition, these "artificial" neural networks operate with high computational and energy costs. In contrast, the biological auditory system processes sound as asynchronous "spikes" (action potentials) that travel through neural substrates with remarkable efficiency. This research addresses the gap between power-hungry, synchronous digital signal processing and the efficient, asynchronous nature of biological hearing. The authors aim to create a system that is not only accurate but also robust to noise and efficient enough to run on low-power, specialized hardware.

Methodology

The proposed system is organized into a two-stage pipeline: a neuromorphic auditory front-end and an SNN classifier.

Neuromorphic Auditory Front-end

The front-end begins with a Cochlear Filter Bank. Unlike standard Mel-scaled filters designed for synchronous computers, this system uses an asynchronous, time-domain cochlear filter bank that better matches the event-driven nature of SNNs. To further refine the signal, the system implements Auditory Masking—inspired by human psychoacoustics—which acts as a biological "attention mechanism." This filters out redundant data and focuses on salient features in both the time and frequency domains.

Finally, Neural Threshold Coding encodes the filtered sound into spatiotemporal spike patterns. By tracking upward and downward crossing events, the system significantly reduces the computational load for the classifier by representing information only when significant changes occur.

The neuromorphic sound recognition pipeline, illustrating the flow from raw audio through cochlear filtering and masking to spike-based classification.

SNN Classifier and Training

The backend utilizes a Spiking Neural Network (SNN) trained with a novel algorithm called MPD-AL (Membrane Potential Dependent Aggregate-Label learning). This algorithm allows the network to learn from the precise timing of spikes, a task traditionally difficult because spike events are non-differentiable. To ensure real-world robustness, the system undergoes Multi-Condition Training, using both clean and noise-corrupted sound samples to improve its resilience to environmental interference.

Key Insights and Results

The system's primary advantage is efficiency through sparsity. Because it is event-driven, it only processes information when there is a change in the signal (a spike), leading to massive energy savings compared to traditional models that process every frame of audio regardless of content. Furthermore, the use of SNNs allows the system to leverage the "membrane potential" of neurons, which naturally tracks the temporal dynamics of speech and sound.

A demonstration GUI showing the system performing continuous spoken digit recognition in real-time, with visualizations of the sound source and SNN output layer.

Key results include:

  • Energy Efficiency: The SNN classifier is cited as being several orders of magnitude more energy-efficient than standard deep learning classifiers.
  • Robustness: Multi-condition training allows the system to maintain high performance even when sound samples are corrupted by background noise.
  • Real-time Interaction: A custom GUI demonstrates the system's ability to perform continuous spoken digit recognition in real-time, highlighting its readiness for practical applications.

Conclusion

This neuromorphic approach represents a milestone in creating energy-efficient, robust sound perception systems. While traditional pattern classification currently dominates due to the maturity of high-performance computing, the transition to large-scale deployment of neuromorphic systems promises significant gains in efficiency. Future work will involve scaling this approach to more complex vocabularies and diverse environmental soundscapes, alongside further optimization of the MPD-AL learning algorithm.

Summary updated

BibTeX
@inproceedings{wu2019robust,
  author = {Jibin Wu and Zihan Pan and Malu Zhang and Rohan Kumar Das and Yansong Chua and Haizhou Li},
  title = {{Robust Sound Recognition: A Neuromorphic Approach}},
  booktitle = {Interspeech},
  year = {2019},
}
Figure