Brain-inspired Action Generation with Spiking Transformer Diffusion Policy Model

Qianhao Wang, Yinqian Sun, Enmeng Lu, Qian Zhang, Yi Zeng

Introduction

In the field of robot learning, Behavior Cloning (BC) serves as a fundamental technique where agents learn to perform tasks by mimicking human demonstrations. However, BC is frequently hampered by "compounding errors," where minor initial deviations lead the robot into unfamiliar states, resulting in task failure. While Diffusion Policies have recently emerged as a robust solution—framing action generation as a denoising process to capture multi-modal distributions—they typically rely on standard Artificial Neural Networks (ANNs).

This paper introduces the Spiking Transformer Modulate Diffusion Policy Model (STMDP). By leveraging Spiking Neural Networks (SNNs), which utilize discrete "spikes" for communication, the authors aim to harness the spatio-temporal processing advantages inherent in biological systems. The STMDP architecture seeks to bridge the gap between the energy-efficient, temporal nature of SNNs and the high-performance generative capabilities of diffusion models.

Methodology

The STMDP framework is built upon the integration of Leaky Integrate-and-Fire (LIF) neurons within a Transformer-based diffusion architecture.

Spiking Neural Network Backbone

Unlike traditional neurons that output continuous activation values, LIF neurons accumulate membrane potential over time. A binary spike is generated only when this potential crosses a predefined threshold. This mechanism allows the model to process information in a manner that is inherently sensitive to temporal dynamics, which is critical for smooth robotic trajectory generation.

Architectural Components

  1. Spiking Transformer Encoder (STE): This module processes visual observations and diffusion timesteps. It utilizes spiking multi-head self-attention to encode environmental context into a latent space.
  2. Spiking Modulate Decoder (SMD): The primary innovation of this work, the SMD, replaces standard decoding layers with a "modulate block." This block applies external conditioning (such as visual features) directly to the cross-attention and feed-forward layers, allowing the spiking framework to effectively integrate conditional information during the denoising process.
  3. Training via Surrogate Gradients: To overcome the non-differentiable nature of discrete spikes, the authors employ surrogate gradients. This technique allows for standard backpropagation by approximating the derivative of the spike function during the training phase.

Experimental Evaluation

The STMDP model was evaluated across four challenging robotic manipulation benchmarks:

  • PushT: A task requiring the robot to maneuver a T-shaped block to a target pose.
  • Can: A high-precision task involving grasping a soda can and placing it into a designated slot.
  • Square: A task requiring the robot to lift a square-holed object and fit it onto a corresponding pillar.
  • ToolHang: A complex assembly task involving a rack and hanging tools.

The researchers compared the STMDP against the standard Transformer-based Diffusion Policy (DPT) using both Denoising Diffusion Probabilistic Models (DDPM) and Denoising Diffusion Implicit Models (DDIM).

Results and Discussion

The experimental results demonstrate that STMDP consistently outperforms traditional ANN-based diffusion models.

Performance Gains

In the Can task, the STMDP achieved a 94% success rate, representing an 8% improvement over the previous state-of-the-art DPT model. Similarly, in the PushT task, the model reached a 75.4% success rate, a 5% increase over standard benchmarks. For the Square task, the STMDP-I variant (utilizing DDIM) achieved the peak performance of 88%.

The Role of Modulation

A key finding of the study is the criticality of the Spiking Modulate Decoder. Ablation studies revealed that removing the modulation block caused success rates in the Can task to plummet from 94% to 22%. This suggests that the specific method of conditioning is vital for SNNs to function effectively within a diffusion framework.

Spatio-Temporal Synergy

The inherent temporal nature of SNNs—processing sequences of spikes rather than static values—appears to provide a "coherence" advantage. This allows the STMDP to generate action trajectories that are more fluid and better aligned with the temporal requirements of physical manipulation.

Conclusion

The STMDP presents a significant step toward brain-inspired robotics, demonstrating that SNNs can be successfully integrated into complex generative models like Diffusion Policies. While the current focus remains on task performance, the use of spiking architectures paves the way for future deployment on low-power neuromorphic hardware. Future research will focus on refining parameter sensitivity in DDIM variants and exploring further architectural optimizations for the modulation modules.

Summary updated

BibTeX
@inproceedings{wang2024brain,
  author = {Qianhao Wang and Yinqian Sun and Enmeng Lu and Qian Zhang and Yi Zeng},
  title = {{Brain-inspired Action Generation with Spiking Transformer Diffusion Policy Model}},
  booktitle = {BICS},
  year = {2024},
  doi = {10.1007/978-981-96-2882-7\_23},
}
Figure