From prediction error to sparse events: Biologically inspired signed-dyadic coding for Mandarin keyword spotting

Zijian Zhou, Rui Liu, Rongxiang Nie, Rui Ma, Ziyao Zhang, Xin Du

TL;DR

Researchers have developed Predictive Signed-Dyadic Coding (PG), a biologically inspired method to make keyword spotting more energy-efficient. By only transmitting "surprising" changes in audio data using a hardware-friendly power-of-two format, they achieved 90% accuracy on Mandarin speech while slashing memory reads and computation cycles by over 30%.

What Problem Does This Paper Address?

Continuous speech recognition, like "Hey Siri" or "OK Google," requires devices to be "always on," which drains battery. While Spiking Neural Networks (SNNs) are great at saving energy in hidden layers by only processing "spikes" of activity, the input interface remains a bottleneck.

Standard systems process every single frame of audio (log-mel features) even if the sound hasn't changed much. This creates redundant memory traffic and computation. The authors aim to bridge this gap by creating an encoder that mimics biological sensory adaptation—essentially "tuning out" predictable sounds and only reacting to significant deviations.

Methods

The core innovation is Predictive Signed-Dyadic Coding (PG). Here is how it works:

  1. Local Prediction: The encoder maintains a local "guess" of what the next audio frame will look like for every channel.
  2. Residual Gating: It calculates the difference (residual) between the actual incoming sound and its prediction. If the difference is small, the system stays silent.
  3. Signed-Dyadic Events: If the residual crosses a learned threshold, the system emits an "event." This event doesn't send the full value; instead, it sends a sign (+ or -) and a dyadic magnitude (a power of two, like $2^{-3}$).
  4. Multiplier-Free Decoding: Because the magnitudes are powers of two, the receiving hardware can reconstruct the signal using only simple bit-shifts and additions—no power-hungry multipliers required.
Figure 1. Predictive signed-dyadic coding and its interface with keyword classification. PG maintains a shared prediction, transmits thresholded signed-dyadic residuals, and reconstructs the input before the Deep-SNN. Temporal aggregation produces the keyword scores.

The researchers tested three placement strategies:

  • DIRECT: Standard dense input (no PG).
  • INPUT-PG: Applying PG only at the initial acoustic input.
  • ALL-PG: Applying PG at the input and after every hidden layer.

Datasets

The study used a Mandarin keyword spotting corpus containing 4,545 utterance clips from 101 different speakers. The task was to identify five specific keyword classes. The data was split into training (3,150 clips), validation (675), and testing (720) sets, ensuring no speaker overlap between groups to test for true generalization.

Key Insights and Ideas

  • The "Less is More" Placement Principle: One might assume applying PG everywhere (ALL-PG) would be most efficient. However, the authors found that INPUT-PG is superior. Hidden layers in SNNs are already sparse; trying to "re-sparsify" them adds more overhead (tracking states and larger packets) than it saves. The real "win" is at the input, where temporal redundancy is highest.
  • Dyadic Alphabet as a Bridge: Using a finite set of power-of-two values provides a clever middle ground between high-precision continuous values and low-precision binary spikes. It preserves enough detail for 90% accuracy while remaining extremely hardware-efficient.
  • Biological Realism: The system effectively implements "predictive processing," a leading theory in neuroscience suggesting that our brains primarily process the errors in our predictions rather than the raw sensory input itself.
Comparison of placement schemes: INPUT-PG targets the high redundancy of the input stream.

Key Results

The authors evaluated their design using both software simulations and a physical hardware routing (Sky130HD library).

  • Accuracy: INPUT-PG reached 90.00% accuracy, slightly outperforming the dense DIRECT baseline (89.58%).
  • Workload Reduction: Compared to the baseline, INPUT-PG reduced:
  • Mean SRAM reads by 33.87%.
  • Inference cycles by 32.77%.
  • Energy Efficiency: In hardware tests on a fixed clip, INPUT-PG reduced incremental inference energy by 39.49%.
  • Comparison: PG significantly outperformed other common encoding methods like Rate coding (which increased workload) and TTFS coding (which lost more accuracy).

Limitations and Future Work

While the software results are robust across 720 clips, the routed hardware energy measurements were performed on a single "frozen" clip. Future work could involve more extensive hardware-level power analysis across diverse acoustic environments.

Additionally, the authors noted that while PG works wonders for correlated inputs, it can add workload if applied to already sparse hidden streams. Refining the "placement criterion" (deciding exactly where to put a PG block in a complex network) remains an open area for optimization in larger, more complex architectures.

Summary updated

Abstract

In continuous spiking keyword spotting, temporally correlated acoustic frames can repeatedly trigger memory access and neural computation at a dense input interface, even when hidden activity is sparse. Predictive processing and sensory adaptation suggest retaining predictable input locally and responding selectively to deviations. Inspired by this mechanism, we propose Predictive Signed-Dyadic Coding (PG). PG maintains a local prediction for each input channel and transmits an event only when the residual exceeds a learned threshold. Each event carries a sign and a finite power-of-two magnitude, and the receiver reconstructs it with fixed-point shifts and additions. In Mandarin keyword spotting, INPUT-PG reaches 90.00% accuracy while reducing mean SRAM reads by 33.87% and inference cycles by 32.77%. PG provides a hardware interface for research on low-power continuous speech processing and event-based sensory systems.
BibTeX
@misc{zhou2026from,
  author = {Zijian Zhou and Rui Liu and Rongxiang Nie and Rui Ma and Ziyao Zhang and Xin Du},
  title = {{From prediction error to sparse events: Biologically inspired signed-dyadic coding for Mandarin keyword spotting}},
  year = {2026},
  eprint = {2610.41381},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41381},
}
Figure