Temporal Dynamics Enhancer for Directly Trained Spiking Object Detectors

Fan Luo, Zeyu Gao, Xinhao Luo, Kai Zhao, Yanfeng LuarXiv 2025 · DOI · arXiv

Abstract

Spiking Neural Networks (SNNs) offer a promising path toward energy-efficient computer vision by mimicking biological neural communication. However, their application to complex tasks like object detection has been hindered by "Temporal Dynamics Pattern Scarcity," where the temporal dimension of the network is underutilized due to redundant input stimuli. This paper introduces the Temporal Dynamics Enhancer (TDE), a framework comprising a Spiking Encoder (SE), an Attention Gating Module (AGM), and a multiplication-free Spike-Driven Attention (SDA) mechanism. TDE significantly improves the diversity of spike firing patterns and achieves state-of-the-art performance on both static (PASCAL VOC) and neuromorphic (EvDET200K) datasets while consuming only 24% of the energy of traditional attention modules.

Introduction: The Challenge of Temporal Scarcity

Spiking Neural Networks (SNNs) are brain-inspired architectures that communicate using discrete "spikes" (1s and 0s) rather than continuous floating-point numbers. This binary nature allows them to be incredibly energy-efficient on neuromorphic hardware. However, SNNs often struggle with high-level tasks like object detection compared to their Artificial Neural Network (ANN) counterparts.

Researchers identified a core bottleneck: Temporal Dynamics Pattern Scarcity. In many current SNN implementations, a static image is simply repeated across multiple time steps, or neuromorphic data is aggregated into highly similar frames. Consequently, neurons receive nearly identical stimuli over time, failing to exploit the SNN's inherent temporal processing capabilities. This leads to "all-silent" or "all-active" spike streams that lack the expressiveness required for accurate object localization and classification.

The TDE Framework

The Temporal Dynamics Enhancer (TDE) is designed to unlock the latent temporal power of SNNs through three primary innovations:

  1. Spiking Encoder (SE): Rather than feeding redundant data at every time step, the SE utilizes a convolutional block to generate varied input stimuli. This ensures that each time step provides a unique "view" of the data, forcing the network to engage its temporal modeling features.
  2. Attention Gating Module (AGM): This module captures dependencies across different time steps. By applying temporal weights to guide the Spiking Encoder, the AGM helps the network focus on the most informative moments while suppressing noise or "unreasonable exploration."
  3. Spike-Driven Attention (SDA): Traditional attention mechanisms, such as those found in Transformers, rely heavily on floating-point multiplications. In neuromorphic hardware, multiplication is significantly more energy-intensive than addition. The SDA uses specialized neuron groups to perform attention using only accumulation (addition) operations, maintaining the network's efficiency and hardware compatibility.
The TDE architecture featuring the Spiking Encoder (a) and the Attention Gating Module (b).

Empirical Evidence: Recovering the "Missing" Spikes

A key insight of this research is the dramatic improvement in spike diversity. Before the application of TDE, many possible spike patterns (e.g., "1000" or "0111") were virtually non-existent in the network's initial layers. By introducing varied stimuli through the Spiking Encoder, TDE effectively "recovers" these missing patterns, as shown in the firing pattern comparison below.

Comparison of spike firing patterns in the first LIF layer with and without TDE, showing a significant increase in pattern diversity.

Performance and Energy Efficiency

The TDE framework was rigorously evaluated on both static and neuromorphic benchmarks:

  • Static Benchmarks: On PASCAL VOC 2007 and 2012, TDE-enhanced models achieved 57.7% mAP@50-95, setting a new record for directly trained SNN detectors.
  • Neuromorphic Benchmarks: On the large-scale EvDET200K dataset, which features event-based camera data with varying illumination and dynamic backgrounds, the framework reached 47.6% mAP.
  • Efficiency: The Spike-Driven Attention (SDA) module proved to be exceptionally efficient, consuming only 0.240× the energy of conventional non-spiking attention modules (TCSA).

Furthermore, TDE is highly versatile. It can be integrated into existing SNN detectors like SpikeYOLO or EMSYOLO with minimal parameter overhead (less than 0.26M additional parameters), providing a consistent performance boost across different base architectures.

Conclusion and Future Directions

The Temporal Dynamics Enhancer successfully addresses the underutilization of time in SNNs. By aligning software architecture with the strengths of neuromorphic hardware—specifically by replacing multiplication with addition and diversifying temporal inputs—the authors have bridged the gap between SNN efficiency and ANN accuracy in object detection.

While the SDA module is highly efficient, its representational capacity on extremely large datasets remains a focus for future improvement. Subsequent research may explore more complex spike-driven relationships and biologically inspired learning paradigms to further exploit the temporal dynamics unlocked by TDE.

Summary updated

BibTeX
@misc{luo2025temporal,
  author = {Fan Luo and Zeyu Gao and Xinhao Luo and Kai Zhao and Yanfeng Lu},
  title = {{Temporal Dynamics Enhancer for Directly Trained Spiking Object Detectors}},
  year = {2025},
  eprint = {2512.02447},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.02447},
}
Figure