Any system that turns sound into a time-frequency picture has to choose how long each frequency channel listens. Long windows separate close harmonics but blur the timing of clicks and consonant onsets. Short windows do the opposite. Ordinary neural networks can now learn this choice by gradient descent, treating window length as a continuous parameter. Spiking neural networks cannot yet do this. These are networks that communicate with binary events in time and run at microwatt power on neuromorphic chips. Their audio front-ends are frozen: a fixed biophysical cochlea model, fixed mel filterbanks, or learned band edges with a fixed window. Every spiking speech result therefore inherits a time-frequency trade-off that nobody optimized for the task.
We propose a learnable spiking cochlea. It is a bank of band-pass filters whose center frequency and duration are trainable. Each filter drives a threshold unit that emits spikes at exact, continuous instants. The front-end is trained end-to-end with a temporally precise spiking network. The key move is how gradients pass through the encoder. We do not use a smooth stand-in for the spike's step function. Instead, we differentiate the spike times exactly using the implicit function theorem. This lets the window-length gradient arrive without the forward/backward mismatch that surrogate methods introduce. The question is not only whether this works. It is whether front-end resolution matters more for spiking networks than for conventional ones, because spiking neurons smooth away fine timing that the encoder fails to deliver.
Core Claim
In a spiking audio pipeline, the effective time-frequency resolution is set by two things together: the front-end window and the low-pass smoothing of downstream leaky integrate-and-fire neurons. Spiking Neural Networks Need High-Frequency Information proves the second is a first-order low-pass filter. That paper compensates with hand-placed max-pooling and depth-wise convolutions in vision models.
We claim that a front-end whose filter durations are learned through exact spike-time gradients will do three things. It will learn shorter high-frequency windows than the same front-end trained inside a conventional network, which is the encoder compensating for downstream smoothing. It will beat both fixed cochlear encodings and surrogate-gradient, time-binned learnable encoders on spiking speech benchmarks. And the gap between learned and best-fixed windows will be larger in the spiking setting than the small gap reported for conventional networks.
That last point matters for an honest framing. In conventional networks, learning one STFT window length moved spoken-digit accuracy only from 79.7% to 80.7% (Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform). If the spiking gain is equally small, the idea is not worth pursuing, and our experiments are built to reveal that.
What Exists and What Is Missing
Three lines of work learn time-frequency tiling in conventional networks, and they supply our parameterizations:
- Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform derives analytic gradients of the STFT with respect to window length.
- Differentiable adaptive short-time Fourier transform with respect to the window length extends this to a per-frame, per-bin window matrix. It shows the matrix becomes erratic without a total-variation penalty.
- Learning filter widths of spectral decompositions with wavelets learns wavelet scales as a first layer, reaching 18.1% phone error on TIMIT. It reports that scale parameters are unusually sensitive to learning rate.
- Learnable Fractional Superlets learns wavelet cycle counts and orders for emotion recognition.
All four run on dense floating-point tensors. None faces the problem we target: the error signal reaching the window parameters must pass through binary events. Their stability findings act as constraints on our design.
On the spiking side, the closest system is sVAD. It learns SincNet band edges in front of a surrogate-gradient spiking recurrent network. However, every filter keeps the same fixed length, and its features are binned into 10–15 ms frames, so window duration is never learned. Other systems keep the front-end fixed:
- The Heidelberg spiking datasets encode audio with a fixed, non-differentiable cochlea and hair-cell model.
- Bullet Trains trains continuous-time spiking networks with exact implicit-function gradients through spike times, taking those fixed spike trains as given. It also shows that rounding spike times to 1–2 ms bins collapses accuracy on a timing-sensitive task.
We reuse the Bullet Trains gradient machinery and push it one stage earlier, into the encoder.
Learning Efficient Auditory Codes Using Spikes Predicts Cochlear Filters motivates the whole proposal. Its result is that asking for a sparse spike code of natural sound produces cochlea-like filters on its own: asymmetric in time, long at low frequencies, short at high frequencies. That model used greedy matching pursuit and Hebbian updates for reconstruction, so it could not be trained for a discriminative task or with a spiking backbone. We ask whether the same kind of structure emerges, and how it differs, when the objective is classification and the pathway is differentiable.
Incorporating Learnable Membrane Time Constant (PLIF) learns neuron leak constants. That changes the cutoff of a low-pass filter, but it provides no band-pass decomposition. We treat it as a competing hypothesis, not a component.
Research Program
The filters. Each of $K$ channels is a causal gammatone-like filter $$h_k(t) = t^{3} e^{-2\pi b_k t}\cos(2\pi f_k t), \quad t \ge 0,$$ with two learnable parameters. The center frequency is $f_k$. The decay rate $b_k$ sets the channel's duration, and therefore its position on the trade-off curve, with duration roughly proportional to $1/b_k$. We parameterize $b_k = f_k / c_k$, where $c_k$ is a learnable cycle count, so the network can choose anywhere from constant-Q (wavelet-like) tiling to constant-bandwidth (STFT-like) tiling. The causal, asymmetric form follows the filters Smith and Lewicki found.
The encoder. The half-wave-rectified filter output drives a leaky integrator $u_k(t)$ with learnable threshold $\vartheta_k$, which resets on firing. The filter output is computed at the audio sample rate and interpolated with a cubic spline, so $u_k$ is smooth between samples. A spike time $t^*$ is the root of $u_k(t^*) = \vartheta_k$, found by Newton's method within the bracketing sample interval. Its gradient with respect to any front-end parameter $\theta \in \{f_k, c_k, \vartheta_k\}$ is exact: $$\frac{\partial t^*}{\partial \theta} = -\left(\frac{\partial u_k}{\partial t}\Big|_{t^*}\right)^{-1} \frac{\partial u_k}{\partial \theta}\Big|_{t^*}.$$ This gradient blows up when the potential only grazes the threshold, so $\partial u_k/\partial t$ is small. This is a known failure mode of exact spike-time methods. We handle it by excluding crossings whose slope is below a fixed fraction of the threshold, and we report how often this happens.
The backbone. The spike events feed a continuous-time spiking network trained with the Bullet Trains scheme. The whole pipeline is therefore differentiable with no surrogate anywhere. Following Leiber et al., a smoothness penalty on $c_k$ across neighboring channels guards against erratic durations. Following Khan and Yener, the front-end parameters use their own learning rate, swept separately.
Analysis. Beyond accuracy, we characterize the learned tiling. We plot learned duration against center frequency. We then compare it against three references: the fixed Heidelberg cochlea, the tiling learned by the identical front-end in a conventional recurrent network, and the Smith–Lewicki filter shapes.
Baselines
Naive reference. The do-nothing reference is the released Spiking Heidelberg Digits and Spiking Speech Commands spike trains from Cramer et al., fed into the identical Bullet Trains backbone with the same width, depth, and training budget. This isolates the front-end as the only change. The reported reference points on SHD are 71.4% for a single-layer recurrent spiking network and 83.2% with optimizations and augmentation, plus 89% for a non-spiking LSTM and 92% for a non-spiking CNN. On SSC, the reported recurrent spiking baseline is 50.9%. No reported Bullet Trains accuracy for SHD or SSC was found on Lacuna, so we will rerun it ourselves under our budget. Beating this baseline means a statistically reliable accuracy gain over five seeds while using no more input channels than the released 700, and preferably far fewer.
Strongest existing method. The strongest existing learnable spiking front-end is the sVAD design: a SincNet encoder with learned band edges and fixed window length, binned into frames and trained with surrogate gradients. We rerun it on the same raw audio with the same backbone capacity, the same epoch budget, and an equal hyperparameter sweep. On QUT-NOISE-TIMIT voice activity detection, sVAD reports 19.1% half-total error rate in high noise with 4.3K parameters. Beating it there means lower error at no more than that parameter count.
On Google Speech Commands v1, the reference is the fully spiking dilated-convolution model on log-mel features from Low-Activity Supervised Convolutional SNNs. It reports 5.5% error at about 5% average spiking activity. Beating it means matching or lowering error without raising spike activity.
Key ablation. The ablation that isolates our ingredient keeps everything identical, including the learnable durations, but bins the encoder spikes at 1 ms and trains the encoder with a fast-sigmoid surrogate gradient. We also run variants with $c_k$ frozen at constant-Q and at constant-bandwidth values. Together these separate two contributions: learning duration, and getting exact gradients to it.
Why not just…?
Why not just add learnable window length to sVAD and train with surrogate gradients? This is the most serious threat. It is exactly our key ablation, and it may work as well as our method. Two pieces of evidence suggest it will not. Bullet Trains shows that 1–2 ms binning destroys performance on timing-sensitive tasks. Hybrid Macro/Micro Level Backpropagation documents that surrogate gradients carry bias and variance, and Khan and Yener show that scale parameters are fragile even under clean gradients. If the binned surrogate variant matches ours in accuracy and in the stability of its learned durations across seeds, we will report that the exact-time machinery is unnecessary. The contribution would then reduce to learnable duration in spiking front-ends.
Why not just keep the hand-designed cochlea and improve the backbone? The Heidelberg cochlea is biophysically well tuned, so it is a strong fixed tiling, not a strawman. The head-to-head with identical backbones settles this. A null result there would mean the fixed cochlea already sits near the task optimum. That would be informative in its own right.
Why not just learn membrane time constants (PLIF) on a fixed front-end? Learning the leak shifts a low-pass cutoff. It cannot create band-pass channels or trade bandwidth against duration per frequency. We test this by running PLIF neurons on the fixed-cochlea input. If PLIF closes most of the gap, the front-end is not where the resolution bottleneck lives.
Why not just train a conventional network with a learnable front-end? For accuracy alone, a non-spiking CNN already reaches 92% on SHD, so our goal is not to beat dense models. The question is the best accuracy available at spiking cost. We report synaptic operations and spike counts alongside accuracy. We also train the same learnable front-end in a non-spiking recurrent network to measure how much the learned tiling differs between the two settings.
Proposed Experiments and Decisive Checks
We will use the raw audio underlying the Heidelberg digits and the Google Speech Commands audio behind SSC, with Google Speech Commands v1 for comparison to framed spiking keyword spotters. QUT-NOISE-TIMIT tests noise robustness from +15 to −10 dB. The Free Spoken Digit Dataset gives a direct comparison to the differentiable STFT's 80.7%. All runs use five seeds, with channel counts of 64 and 128.
Check 1: is the gain bigger in spiking networks? We measure the learned-minus-best-fixed accuracy gap for the spiking pipeline and for its conventional twin. Our claim predicts a gap clearly larger than the roughly one-point conventional gain reported by Leiber et al. A similar gap falsifies the idea that resolution matters more for spiking networks.
Check 2: does the encoder compensate for downstream smoothing? We sweep the backbone's membrane time constant upward, which strengthens its low-pass smoothing. The low-pass-compensation hypothesis predicts that learned high-frequency durations shrink as smoothing increases. The conventional twin should show no such trend.
Check 3: does timing precision matter in the encoder? We compare exact spike times against 1 ms and 0.25 ms binning with surrogate gradients. We look at both accuracy and across-seed variance of the learned $c_k$. Exact gradients should give tighter, more consistent tilings, especially in the noisy QUT conditions where transients carry the signal.
Stability report. Throughout, we log the fraction of grazing-crossing spikes excluded by the slope cutoff. If that fraction is large, the exact-gradient advantage is partly illusory, and we will say so.
References
- Spiking Neural Networks Need High-Frequency Information
- Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform
- Differentiable Adaptive Short-Time Fourier Transform with Respect to the Window Length
- Learning filter widths of spectral decompositions with wavelets
- Learnable Fractional Superlets with a Spectro-Temporal Emotion Encoder for Speech Emotion Recognition
- SVAD: A Robust, Low-Power, and Light-Weight Voice Activity Detection with Spiking Neural Networks
- The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks
- Bullet Trains: Parallelizing Training of Temporally Precise Spiking Neural Networks
- Learning Efficient Auditory Codes Using Spikes Predicts Cochlear Filters
- Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networks
- Low-Activity Supervised Convolutional Spiking Neural Networks Applied to Speech Commands Recognition
- Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks.