Echo Suppression as Spike Suppression: Adaptive Inhibition for Low-Energy Sound Localization in Reverberant Rooms

Locating a sound source with a small microphone array means measuring tiny timing differences between channels. In a real room, wall reflections create delayed copies of the sound that arrive from the wrong directions and smear those timing cues. Spiking neural networks are networks whose neurons communicate with sparse binary pulses, so they spend energy roughly in proportion to how many pulses they emit. They are attractive for always-on localization on hearing aids, robots, and wearables. The most complete spiking localizer to date, Spike-based Neuromorphic Model for Sound Source Localization, turns raw audio into pulses with resonating neurons instead of a Fourier transform. It reports 0.33° mean absolute error on the SLoClas benchmark at about 9.58 µJ per inference. It was tested only against added noise, never against reverberation.

This proposal starts from an observation about event-driven hardware. In a pulse-based encoder, a reverberant tail keeps the frequency-tuned neurons ringing and firing long after the direct sound has passed. Reverberation should therefore cost a spiking localizer twice: it adds wrong-direction evidence, and it adds pulses that each cost energy. If that is true, suppressing echoes inside the network should improve accuracy and lower energy at the same time. We propose to build that suppression from two biological mechanisms. The first is a delayed inhibitory input at the timing-cue stage, modeled on the precedence effect (listeners lock onto the first-arriving wavefront and ignore later echoes). The second is slow, room-dependent adaptation of that inhibition's delay, modeled on how auditory cortex retunes to larger rooms.

Core Claim

Our claim is that a spiking localizer gains robustness to reverberation, and reduces its spike count, when its interaural timing detectors receive a learned, frequency-specific, delayed inhibitory input whose delay adapts online to the room's decay rate. The key point is that this suppression must act on the timing cues before evidence is integrated over time. Integrating afterwards cannot do the same job.

The argument for that placement is short. The short-term memory in the existing spiking model accumulates cues across adjacent windows so that consistent sources build up and transient noise decays (Spike-based Neuromorphic Model for Sound Source Localization). Early reflections, however, are not transient. A reflection off a fixed wall arrives from a fixed direction on every utterance, so accumulation reinforces these ghost sources rather than rejecting them. Two lines of evidence show that the useful information sits in timing relative to the onset. Modeling the precedence effect for binaural sound source localization in noisy and echoic environments shows that measuring cues at envelope peaks, combined with a signal-triggered inhibition window, localizes reliably in a 750 ms conference room. That system is hand-crafted signal processing with no learning and no energy accounting. Cortical adaptation to sound reverberation shows that ferret cortical neurons undo reverberation with an excitatory receptive component followed by extended inhibition. That inhibition shifts about 9.3 ms later in large rooms than in small ones, lasts longer at low frequencies, and readjusts within seconds of a room change. That study is monaural neuroscience with no engineered network. Our proposal takes the precedence mechanism as the circuit, the cortical finding as the adaptation rule, and the resonate-and-fire spiking localizer as the substrate. We then test the claim that the energy and accuracy benefits come from the same mechanism.

Mechanism

We keep the front end of the existing model: resonate-and-fire neurons (spiking neurons with an internal oscillation that makes each one respond to a narrow frequency band) combined with phase-locking coding. Into each frequency channel's coincidence detectors, the neurons that compare spike timing across microphones, we add an inhibitory synapse. It is driven by a delayed, low-pass-filtered copy of that channel's own onset spikes. For coincidence neuron $i$ in frequency channel $f$, the membrane potential $v$ evolves as

$$ v_i[t+1] = \beta\, v_i[t] + \sum_j w_{ij}\, s_j[t] - g_f \sum_{k=0}^{K} h_f(k)\, e_f[t - d_f - k], $$

where $\beta$ is the leak factor, $s_j$ are the phase-locked input spikes with weights $w_{ij}$, and $e_f$ is a binary onset-event train for channel $f$. The inhibitory kernel $h_f$ has a learned width, and it is applied with gain $g_f$ after delay $d_f$. The neuron fires and resets when $v_i$ crosses threshold, as in standard leaky integrate-and-fire models. Every term is an accumulate operation, so no multipliers are needed.

The delay $d_f$ and kernel width are not fixed constants. A small population of slow "room-state" neurons integrates how quickly channel energy decays after offsets, which is a spike-domain proxy for reverberation time. Through a learned lookup, this population shifts $d_f$ and the kernel width. All parameters, including the lookup, are trained end to end with surrogate gradients, the standard workaround for the non-differentiable spike. We will then inspect whether the learned delays lengthen with room size and at low frequencies, as the ferret data predict. That inspection is a sanity check on the mechanism, not the main result.

Research Program

The first step is diagnostic and could end the energy half of the story early. We will retrain the published architecture of Spike-based Neuromorphic Model for Sound Source Localization and run it on reverberant versions of its training data. These are made by convolving SLoClas and TIMIT sources with simulated room impulse responses, generated with the gpuRIR simulator used by SRP-DNN, at reverberation times from roughly 200 to 900 ms. We will measure how error and total spike count scale with reverberation time. If spike count does not rise appreciably with reverberation, the double-cost premise fails. In that case the project continues as a pure robustness study and we say so.

The second step adds the inhibitory pathway in three escalating forms: fixed delay, learned but static delay, and room-adaptive delay. All three are trained under the same multi-condition reverberant regime. The third step tests on real recordings. These are LOCATA Task 1 and Task 3 (The LOCATA Challenge), restricted to the four-microphone hearing-aid configuration and the NAO robot head, because those match the compact-array setting where the energy budget matters. The fourth step, contingent on hardware access, deploys the best model on Loihi-class hardware. Earlier robotics work there reported energy gains of more than two orders of magnitude over embedded GPUs (Tang). Until that deployment happens, all energy claims are stated as synaptic operation counts using the same accounting as the 9.58 µJ figure. This is the emulation gap the research file flags, and we will not blur it.

Baselines

The naive reference point is the published spiking localizer trained only on its original non-reverberant data and tested on reverberant data. It shows how badly reverberation hurts out of the box. Its reported SLoClas result, 0.33° mean absolute error at about 9.58 µJ per inference (Spike-based Neuromorphic Model for Sound Source Localization), is the anchor for the low-reverberation end of our sweep. We must reproduce it within reasonable tolerance before any comparison counts.

The strongest spiking competitor, and the most important comparison, is the same architecture retrained on exactly our multi-condition reverberant data. It gets the same epochs, the same hyperparameter search budget, and an added parameter count equal to our inhibitory pathway, placed as extra excitatory capacity. This isolates whether explicit delayed inhibition does anything that more reverberant training data and capacity cannot. Beating it means lower azimuth error at reverberation times above 500 ms, the regime where the file notes heuristic methods fail. It also means equal or lower spike count, with both differences holding across at least three seeds.

The strongest dense competitor is the causal convolutional-recurrent network of SRP-DNN, which learns direct-path phase differences. It reports lower miss-detection and false-alarm rates than SRP-PHAT and other baselines on LOCATA, but no reported numerical result for SRP-DNN on LOCATA Task 1 azimuth error was found on Lacuna. We will retrain it, in single-source mode, on our data with matched tuning budget, and also evaluate an 8-bit quantized version. On LOCATA Task 1 the reference ceiling is that top challenge entries achieved roughly 1.0° average azimuth error (The LOCATA Challenge). We do not expect to beat that ceiling. Our success criterion is to come within a stated margin of it, which we will fix before running, while using fewer synaptic operations than the quantized network's multiply-accumulates. Both quantities are reported in counts, not converted to joules.

The classical competitor is direct-path cross-correlation estimated from linear-prediction coefficients (Direct-Path Signal Cross-Correlation Estimation for Sound Source Localization in Reverberation). It reports 92.24% frame accuracy at 1 m and 81.35% at 3 m in a 400 ms room. That custom recording set is not available to us, so we will reimplement the method and run it on our simulated sweep and on LOCATA.

The ablations isolate the key ingredient. One ablation keeps the inhibitory pathway but freezes its delay at the average learned value; it separates adaptation from inhibition. A second makes the delay frequency-independent. A third moves the identical inhibition to after the short-term memory stage; it tests the claim that placement before integration matters.

Why not just…?

Why not just train the existing spiking localizer on reverberant data? This is the real threat. A surrogate-gradient network with recurrent memory might learn something equivalent to delayed inhibition from its excitatory weights alone. The capacity-matched multi-condition baseline is designed to settle this. If it matches our model on both error and spike count at long reverberation times, the explicit mechanism is unnecessary and we will report that. We expect it to fall short only on spike count and on rooms not seen in training. Excitatory-only training can down-weight echo cues downstream, but it cannot stop the echo spikes from being generated, and a fixed learned solution has no way to retune to a new room.

Why not just put a classical precedence or dereverberation front end before the spiking network? The precedence model of Modeling the precedence effect and linear-prediction dereverberation both work. However, they run as continuous signal processing, and the file identifies continuous front ends as the energy bottleneck of spiking audio pipelines. We will build this hybrid as an extra baseline and count its multiply-accumulates alongside the network's accumulates. The hybrid could win on accuracy. Our claim is narrower: it will lose on total operations at comparable accuracy.

Why not just quantize a small dense network and run it on a microcontroller? An 8-bit causal recurrent network is the practical incumbent and may well win in measured joules on commodity chips, because spiking advantages only materialize on event-driven silicon. We address this by reporting operation counts for both. We also measure how the spiking model's activity falls during speech pauses, while the dense network recomputes every frame. We state plainly that the energy conclusion is conditional on neuromorphic deployment until step four is done.

Why not just turn the robot's head toward the source? Head orientation helped speech recognition in Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization. It does not remove the need for an initial direction estimate in a reverberant room, and it does not apply to hearing aids or fixed arrays. It complements this work rather than replacing it.

Decisive Checks

Three outcomes decide the project. First, the diagnostic must show that the baseline spiking localizer's spike count grows with reverberation time; without that, the shared energy-accuracy mechanism is false. Second, on the reverberation sweep above 500 ms and on LOCATA Tasks 1 and 3, the adaptive inhibitory model must beat the capacity-matched multi-condition baseline on azimuth error while not exceeding its spike count. The variant with inhibition moved after integration must do worse, which confirms that placement matters.

Third, room adaptation will be tested on streams that splice impulse responses from a small and a large room mid-utterance. We will measure how long localization error takes to recover after the switch, comparing the adaptive and frozen-delay models. If adaptation does not shorten recovery, the cortical component adds nothing, and the proposal reduces to a learned precedence circuit. That would still be a useful result, but a smaller one.