TSFFormer: A Frequency-Domain Spike Transformer for Green Human Activity Recognition

Jiale Deng, Shuiying XiangarXiv 2026 · arXiv

TL;DR

TSFFormer is a novel, purely spiking neural network architecture that achieves state-of-the-art accuracy in Human Activity Recognition (HAR) while maintaining an ultra-low energy footprint of just 0.1 pJ per operation. By combining frequency-domain Fourier attention with a biologically inspired, dual-compartment adaptive neuron, it breaks the traditional trade-off between the high accuracy of deep learning and the extreme efficiency of neuromorphic computing.

What Problem Does This Paper Address?

Human Activity Recognition (HAR) using wearable sensors (IMUs) is essential for health monitoring and sports analytics. However, current solutions face an "arithmetic-latency deadlock."

Traditional Deep Neural Networks (DNNs) are accurate but rely on dense floating-point Multiply-Accumulate (MAC) operations that drain battery life on edge devices. Spiking Neural Networks (SNNs) offer a "green" alternative by using sparse, energy-efficient Accumulate (AC) operations, but they historically struggle with three issues:

  1. Spatial Blind Spots: They often flatten sensor data, losing the physical relationship between different body parts.
  2. Neural Rigidity: Standard spiking neurons use fixed firing thresholds, making them poor at adapting to the varying rhythms of human movement.
  3. Gradient Degradation: Training deep SNNs over long sequences is notoriously difficult, often leading to slow convergence or poor performance.

Methods

The authors introduce TSFFormer, a purely spiking Fourier Transformer. Its architecture is built on three main pillars:

  • MultiHeadFourier Attention: Instead of the standard dot-product attention (which has $O(V^2)$ complexity), this module performs global mixing in the frequency domain using the Fast Fourier Transform (FFT). This reduces complexity to $O(V \log V)$, saves ~50% GPU memory, and is naturally suited for capturing the periodic patterns inherent in human motion.
  • TSLIFNode (Temporal Spiking LIF): Inspired by the dual-compartment (dendrite and soma) structure of biological neurons, this node uses dynamic thresholds. It adjusts its firing sensitivity based on historical activity—lowering the threshold during intense movement to capture detail and raising it at rest to suppress noise.
  • Topology-Preserving Embedding (MS-SPS): Unlike models that flatten data, TSFFormer uses a non-downsampling patch embedding to preserve the spatial structure of sensor nodes across the body.
Figure 1. The overall structure of our proposed TSFFormer framework.

Datasets

The model was evaluated on 8 diverse public IMU benchmarks to ensure robustness across different scenarios:

  • Multi-node: PAMAP2, TNDA-HAR, and HuGaDB (large-scale, asymmetric sensor nodes).
  • Complex/Daily Activity: WISDM and OPPORTUNITY.
  • Single-node: USC-HAD.
  • Clinical/Elderly: HAR70 (specifically used to test the adaptive thresholding on non-stationary rhythms).

Key Insights and Ideas

  • Frequency Domain is a Natural Fit for HAR: Human activities like walking or running are inherently periodic. By moving attention to the frequency domain, the model captures global dependencies more efficiently than spatial-only transformers.
  • SNNs Can Beat ANNs: One of the most significant findings is that TSFFormer doesn't just match SNN performance; it outperforms the classic DeepConvLSTM (an Artificial Neural Network) on several datasets. This challenges the assumption that SNNs are inherently less capable than their non-spiking counterparts.
  • Adaptive Thresholds Solve Gradient Issues: The dual-compartment TSLIF neuron acts as a powerful regularizer. By dynamically modulating thresholds, it mitigates the gradient vanishing/explosion problems common in deep SNN training, leading to a "cliff-like" drop in loss during early training stages.
Biological inspiration for the TS-LIF model and its time-series decomposition capabilities.

Key Results

TSFFormer establishes new state-of-the-art (SOTA) benchmarks for spiking networks across all 8 tested datasets.

  • Accuracy: On the WISDM dataset, it achieved 86.35% accuracy, a massive 33.92% leap over the next best SNN (SpikeTCN). It also outperformed the ANN baseline DeepConvLSTM (71.23%).
  • Efficiency: The model operates with an energy footprint of 0.1 pJ, compared to the ~4.6 pJ required for standard FP32 MAC operations in DNNs.
  • Training Speed: TSFFormer converges significantly faster than traditional models. As shown in the training dynamics comparison, it reaches a 50% accuracy threshold in just 5 epochs, whereas DeepConvLSTM requires 35 epochs—a 7x speedup in convergence.
Comparison of training dynamics: TSFFormer shows much faster convergence and higher accuracy ceilings than DeepConvLSTM.

Limitations and Future Work

While TSFFormer is highly effective, the authors noted a few areas for improvement:

  • Single-Node Redundancy: On datasets with only a single sensor (like USC-HAD), the global frequency mixing can introduce slight redundancy, as there is no complex spatial topology to map.
  • Computational Complexity of TSLIF: The dual-compartment neuron is more complex than a standard LIF neuron. While it improves accuracy and convergence, it adds a small amount of per-epoch computation time.

Future work may involve migrating this frequency-spiking hybrid structure to other multimodal time-series tasks beyond IMU data, such as EEG or financial forecasting.

Summary updated

Abstract

Wearable Inertial Measurement Units (IMUs) enable pervasive Human Activity Recognition (HAR), but traditional Deep Neural Networks (DNNs) suffer from an "arithmetic-latency deadlock" due to dense floating-point MAC operations. While Spiking Neural Networks (SNNs) offer ultra-low power, existing ones face spatial blind spots, rigid neurons, and gradient degradation. To address these, we propose TSFFormer, a purely spiking Fourier Transformer for Green HAR. It introduces two key innovations: (1) a MultiHeadFourier attention that performs frequency-domain global mixing, reducing complexity from \(O(V^2)\) to \(O(V\log V)\), eliminating spatial blind spots, and saving \(\sim\)50% GPU memory; and (2) a dual-compartment Temporal Spiking LIF (TSLIFNode) that utilizes dynamic thresholds to adapt to non-stationary rhythms and mitigate gradient degradation. Extensive experiments on 8 public IMU benchmarks demonstrate that TSFFormer surpasses state-of-the-art SNNs and even the classic DeepConvLSTM, achieving a \(\sim\)3\(\times\) faster training speed with an energy footprint of 0.1 pJ. Ablation studies confirm the synergistic effect of our modules, providing a highly accurate, ultra-low-power event-driven paradigm for wearable sensing.
BibTeX
@misc{deng2026tsfformer,
  author = {Jiale Deng and Shuiying Xiang},
  title = {{TSFFormer: A Frequency-Domain Spike Transformer for Green Human Activity Recognition}},
  year = {2026},
  eprint = {2611.12090},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12090},
}
Figure