A low-power sound detector cannot afford to listen to every frequency, so it has to decide which frequency channels to keep before it turns the signal into the binary pulses ("spikes") that a neuromorphic chip computes with. A neuromorphic chip is a processor that does work only when such a pulse arrives. The established rule for choosing channels comes from sampling theory: keep the bands with the highest signal-to-noise ratio (SNR) and drop the rest before sampling. Once noise is sampled it folds onto the signal and cannot be removed later. This rule was built to preserve the signal on average, and it works well for recognizing what a sound is. We argue that it is the wrong rule for detecting when a sound starts and stops. Onsets are short, often broadband bursts of change. The bands that are loudest relative to steady noise are frequently not the bands where the onset stands out sharply above noise fluctuations.
The proposal is to select channels by "transient SNR", meaning how much the onset change in a band exceeds the noise's own frame-to-frame jitter. Selection happens before spike encoding in a spiking Transformer that predicts frame-level event activity and onsets. The same selection, expressed as a binary input mask, becomes the main way a frozen, clip-classification-pretrained spiking backbone is adapted to dense boundary prediction. The concrete, falsifiable prediction is that, at a fixed spike and channel budget under noise, stationary-SNR selection improves clip-level detection scores, while transient-SNR selection improves boundary-sensitive scores. If no such split appears, the idea is wrong.
Core Claim
Classification and localization pull apart in audio. In the DCASE 2022 baseline analysis, adding Audio Spectrogram Transformer embeddings raised the detection-oriented PSDS-2 score from 0.536 to 0.722, but the localization-oriented PSDS-1 barely moved, from 0.336 to 0.351 (Ronchini et al.). We hypothesize that the same split exists one level earlier, in the front end. A channel-selection criterion is implicitly an objective, and the classical criterion optimizes the wrong one for boundaries. Concretely, for frequency channel $c$ with band envelope $e_c(t)$, we define
$$\rho_c = \frac{\mathbb{E}\left[(\Delta e_c(t))^2 \mid t \in \mathcal{O}\right]}{\mathrm{Var}\left[\Delta e_c(t) \mid t \in \mathcal{B}\right]},$$
where $\Delta e_c(t) = e_c(t) - e_c(t-1)$ is the frame-to-frame envelope change, $\mathcal{O}$ is the set of frames within a short window of annotated onsets, and $\mathcal{B}$ is the set of background-only frames. The rule keeps the top $k$ channels by $\rho_c$ under a fixed channel budget. The stationary competitor ranks channels by event energy over background energy, which is the spirit of water-filling (Shannon Meets Nyquist, Chen et al.). An online variant estimates only the denominator from recent frames the detector judges inactive, and keeps the numerator from training statistics. This makes reselection possible when the noise changes, which addresses the non-stationarity weakness of offline band pruning.
Why do this before spiking rather than letting the network ignore bad channels? A noisy channel fed to a spike encoder emits spurious spikes. Those spikes cost energy on every downstream layer and get integrated into membrane potentials before any attention mask can act. This is the spiking analogue of noise folding: the corruption is committed at encoding time.
Where This Sits
Three works motivate the front-end idea and show what is missing. Selective Gammatone features keep 12 of 36 channels with the lowest variance under noise, and raise 0 dB car-noise recognition from 40.08% (MFCC) to 81.03% (Leng et al.). However, the selection is offline, the back end is an HMM, and the task is isolated-clip recognition. An adaptive analog front end powers only about 5 of 13 sub-bands chosen by real-time SNR and runs keyword spotting at about 230µW (He et al.), but it detects keywords coarsely and never localizes boundaries. A resonate-and-fire spiking front end replaces the Fourier transform and prunes frequencies with a learned binary mask for sound localization at 9.58µJ (Zhang et al.). Its frequency pruning is learned after encoding, and its target is direction of arrival, not event timing. All three select channels for recognition or stationary quality. None asks which channels carry onset information.
The spiking back end draws its method from two papers. Softmax-free spiking self-attention (Spikformer) and its Hadamard-product simplification (Jiang et al.) make attention addition-only. Both were evaluated only on image and gesture classification, so dense temporal output is untested. Jiang et al. also supply a constraint: they show that stacking two masks in attention is redundant. That result predicts that our pre-spike gate combined with a learned post-spike frequency mask may also be redundant, and we test this directly rather than assume otherwise.
The boundary objectives are reused from three sources. We use the onset-conditioned hierarchy, which lifted URBAN-SED event F-score from 7.34% to 23.20% (Pankajakshan et al.). We use the confidence-regularized loss, which stops gradients on confident frames so learning concentrates on transitions (Choi and Chang). The offset-regression head idea comes from HEiMDaL.
The adaptation step borrows from dense-prediction transfer. Regularized mask tuning adapts a frozen model by learning binary masks over attention weights (Zheng et al.). Prompt-plus-convolutional-stem tuning restores spatial detail lost by classification pretraining (Ma et al.). Binary masks suit spiking hardware because they add no multiplications at inference, unlike adapters. Our addition is to treat the input-channel gate itself as part of the task mask. The tagging task and the boundary task then share one frozen backbone but listen to different channels.
Research Program
We will build a spiking detector in four stages. The first is a 64-channel Gammatone filterbank with envelope extraction. The second is a channel gate that keeps $k \in \{8, 16, 32, 64\}$ channels. The third is delta spike encoding, where a channel spikes when its envelope changes by more than a threshold. The fourth is a Hadamard spiking Transformer over time frames that keeps full frame resolution, with no temporal pooling. Output heads predict class-agnostic activity, onsets, and class-wise activity in the onset-conditioned order, and are trained with confidence-regularized entropy.
The backbone is first pretrained for clip-level tagging on DESED weak labels with all channels on. It is then adapted to frame-level boundaries in one of four ways: full fine-tuning, attention-mask tuning only, input-gate reselection only, or both.
Energy is reported as synaptic operations converted with the same per-accumulate figures used by the spiking-attention papers. We state plainly that this is a simulation-level estimate; physical-chip measurement is out of scope and remains an open limitation.
The analysis has three parts. First, we measure how much the two channel rankings overlap per noise type, and which frequency regions each favors. Second, we plot PSDS-1 and PSDS-2 against the channel budget $k$ for each criterion. Third, we break down onset error separately for impulsive classes (dishes, door) and tonal classes (alarm, speech). We expect the transient criterion to help most on impulsive classes, and to help little or slightly hurt on tonal ones.
Baselines
The naive reference keeps all 64 channels with no gate and uses the same spiking backbone and heads. It sets the ceiling on information and the floor on efficiency, and it tells us whether any selection helps under noise. As the file notes, selection is expected to cost a little in clean conditions.
The strongest conventional reference on DESED is the DCASE 2022 CRNN baseline, which reports PSDS-1 of 0.336 and PSDS-2 of 0.536 (Ronchini et al.). We will also include the CRNN with axis-wise attention and confidence-regularized entropy, which reports Event-F1 of 51.11 against 49.17 for binary cross-entropy (Choi and Chang). On URBAN-SED the reference is the onset-conditioned joint model at 23.20% event F-score (Pankajakshan et al.). These are the strongest competitors because they already target boundaries in non-spiking models. We will retrain them on our noise-remixed splits with the same training data and epochs and a matched hyperparameter search of equal trial count, and report their multiply-accumulate counts next to our operation counts. We found no reported result on Lacuna for any spiking sound event detector on DESED or URBAN-SED. The spiking comparison is therefore internal, and the fair claim is that boundary quality approaches the CRNN at a fraction of its operations, not that it beats it outright.
The key ablation is stationary-SNR selection at the same $k$, implemented with Leng et al.'s variance criterion and an energy-ratio water-filling ranking, with everything else identical. Beating it means a higher PSDS-1 and event-F1 at equal spike count, with PSDS-2 within noise of the stationary gate. A second ablation replaces the pre-spike gate with a learned post-encoding frequency mask in the style of Zhang et al., matched on total spikes. This isolates the "before encoding" part of the claim.
Why not just…?
Why not just keep all channels and let a learned attention mask drop the noisy ones? This is the most serious threat, because a learned mask is optimized directly for the loss and could discover transient-informative channels without any hand-designed criterion. Our argument rests on what happens before the mask acts: noisy channels spend spikes and perturb membrane states earlier in the network. That argument only matters at a fixed spike budget. The post-encoding-mask ablation, matched on total spikes, settles it. If the learned mask equals the transient gate at equal energy, the "pre-spike" half of the claim fails, and we will report the transient criterion only as an interpretable initialization for the learned mask.
Why not just run a tiny quantized CNN on a microcontroller? A 13,832-parameter network already localizes wake-words well (HEiMDaL), and our energy figures are simulated, not measured. We therefore do not rest the contribution on spiking efficiency. We also apply both selection criteria to the input of the conventional CRNN. If transient selection helps the CRNN's PSDS-1 too, the criterion result is general and the spiking implementation is one deployment of it. If it helps only the spiking model, that points to the encoding-time corruption mechanism.
Why not just fine-tune the pretrained backbone end to end for boundaries? Full fine-tuning is the obvious way to recover frame resolution. Our adaptation arms include it at the same data and step budget. Mask-based adaptation earns its place only if it matches full fine-tuning on PSDS-1 while storing one frozen backbone plus small binary masks per task. Masks have outperformed heavier tuning in vision (Zheng et al., Ma et al.), but this has not been shown for spiking or audio models, so the outcome is genuinely open.
Why not just smooth predictions with a median filter after frame-wise classification? Frame-wise classification with smoothing is the 7.34% event-F URBAN-SED baseline (Pankajakshan et al.). It is included as the naive head. Smoothing cannot recover onsets that the front end never encoded, which is exactly the failure our gate addresses.
Proposed Experiments
The main experiment remixes DESED synthetic soundscapes and URBAN-SED at 10, 0, and −5 dB with stationary noise (car), babble-like noise, and impulsive non-stationary noise. It then sweeps $k$ for the transient, stationary, and learned-mask gates, and reports PSDS-1, PSDS-2, event-F1 with a 200ms collar, and synaptic operations.
The decisive check is a crossing pattern. For transient selection over stationary selection, the PSDS-1 gain should be positive while the PSDS-2 change is near zero or negative, and this pattern should grow as SNR falls. A uniform improvement or no difference on both scores falsifies the claim that the criteria trade off.
A second check tests online reselection. We switch noise type mid-stream in 60-second clips, where long-form degradation is already documented (Turpault et al.). We then measure how quickly onset error recovers relative to a frozen offline gate.
A third check tests cross-task and cross-domain transfer. We adapt the tagging-pretrained spiking backbone from synthetic to real DESED recordings and compare the four adaptation arms on PSDS-1. The question is whether reselecting input channels alone recovers a meaningful share of the boundary gap left by tagging pretraining.
References
- Description and Analysis of Novelties Introduced in DCASE Task 4 2022 on the Baseline System
- Shannon meets Nyquist: Capacity limits of sampled analog channels
- Channel Capacity under General Nonuniform Sampling
- Selective gammatone filterbank feature for robust sound event recognition
- An Adaptive Multi-Band System for Low Power Voice Command Recognition
- Spike-based Neuromorphic Model for Sound Source Localization
- Spikformer: When Spiking Neural Network Meets Transformer
- Efficient Transformer Attention for SNNs via Hadamard Simplification
- Onsets, Activity, and Events: A Multi-task Approach for Polyphonic Sound Event Modelling
- Confidence Regularized Entropy for Polyphonic Sound Event Detection
- HEiMDaL: Highly Efficient Method for Detection and Localization of Wake-Words
- Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Models
- When Visual Prompt Tuning Meets Source-Free Domain Adaptive Semantic Segmentation
- Sound Event Detection and Separation: A Benchmark on Desed Synthetic Soundscapes