DAQ: Decision-Aligned Post-Training Quantization for Spiking Transformers

Jialin Chen, Xiaochen Zhao, Aili WangarXiv 2026 · arXiv

TL;DR

DAQ (Decision-Aligned Post-Training Quantization) is a novel calibration method designed to compress Spiking Transformers into 4-bit precision for both weights and internal membrane variables. By aligning the quantized model's firing decisions with a high-precision reference, DAQ slashes energy consumption by up to 66% and significantly reduces memory overhead while maintaining near-lossless accuracy across classification, segmentation, and tracking tasks.

What Problem Does This Paper Address?

Spiking Transformers (STs) are the "green" alternative to traditional Transformers, using sparse, discrete "spikes" to communicate rather than continuous values. However, they have a hidden cost: their synaptic weights and "membrane variables" (the internal state of each neuron) still require massive amounts of high-precision storage and computation.

While Post-Training Quantization (PTQ)—reducing bit-depth without full retraining—is a standard solution, it struggles with spiking neurons. Because these neurons operate on a "fire or don't fire" threshold, even a tiny quantization error can cause a neuron to miss a spike or fire an extra one. In recurrent models, these errors don't just stay local; they propagate and accumulate over time, leading to a "butterfly effect" that degrades model performance.

Methods

The authors introduce Decision-Aligned Post-Training Quantization (DAQ). Instead of the standard approach of minimizing the Mean Squared Error (MSE) between original and quantized values, DAQ focuses on the consequences of those values.

Key components include:

  • Decision-Region Alignment: The method analyzes "decision boundaries"—the specific voltage levels where a neuron switches from silent to firing. It calibrates quantizers to ensure the quantized membrane potential stays on the same side of the boundary as the original floating-point model.
  • Temporal Error Analysis: The researchers derived conditions for "firing preservation," accounting for how errors in the internal state (the membrane potential) at time step $t$ affect firing at time step $t+1$.
  • Decision-Aligned Objective: They formulated a loss function using a one-sided Huber penalty. This specifically penalizes quantization errors that push a potential across a firing threshold while ignoring errors that keep the potential safely within its "correct" decision region.
  • Quantized Trajectory Calibration: Calibration is performed along the quantized model’s own state trajectory, ensuring the model learns to correct for the specific types of errors it will encounter during actual inference.
Figure 2. Overview of DAQ. The frozen floating-point model provides reference firing decisions, while the quantized model follows its own state trajectory. At shared sampled positions, a weighted decision-aligned loss encourages quantized potentials to remain within the reference decision regions. Event losses are averaged within and across groups to calibrate the weight and membrane quantizers.

Datasets

The authors evaluated DAQ across a diverse range of tasks and architectures:

  • Image Classification: ImageNet-1K, CIFAR-10, and CIFAR-100.
  • Semantic Segmentation: ADE20K.
  • Event-Based Object Tracking: FE108, FELT, and VisEvent.

Key Insights and Ideas

  • Directionality Matters: In a spiking neuron, a positive error is much more dangerous if the neuron is already near its firing threshold (it might trigger a "false" spike), whereas a negative error is more dangerous if it's just above the threshold (it might "miss" a spike). DAQ treats these directions asymmetrically.
  • The "Silent" State Trap: Two neurons might both be silent at a given moment, but if their internal membrane potentials differ due to quantization, they are effectively "primed" differently for the next time step. DAQ's temporal analysis ensures these hidden state differences don't lead to divergent behavior later.
  • Efficiency via Sparsity: By successfully quantizing membrane variables to 4 bits (U4), the researchers halved the memory footprint for recurrent states in certain architectures, which is critical for hardware with limited on-chip memory.
Figure 3. Decision boundaries and temporal propagation of quantization errors. (a) Firing preservation depends on the reference potential and the direction of the perturbation relative to the decision boundary. (b) Matching non-firing outputs can leave different post-reset states and lead to a later firing mismatch.

Key Results

DAQ demonstrates that high-performance Spiking Transformers can run efficiently on low-bit hardware:

  • ImageNet-1K: Achieved 83.4% top-1 accuracy using 4-bit weights and 4-bit membrane variables (W4/U4), outperforming existing PTQ baselines like BRECQ.
  • Energy Efficiency: On the FE108 tracking benchmark, DAQ reduced energy consumption by 66.2% compared to the floating-point version.
  • Memory Savings: For recurrent models like SDT-8-768, DAQ reduced the combined parameter and buffer storage from 93.4 MiB to 47.5 MiB.
  • Robustness: The method maintained competitive performance even on complex tasks like semantic segmentation, where it achieved 38.9% mIoU on ADE20K with a 44% reduction in energy per image.
Figure 1. Accuracy versus energy on ImageNet-1K. DAQ achieves 83.4% top-1 accuracy with 4-bit weights and membrane variables (W4/U4). Bubble area represents parameter storage.

Limitations and Future Work

  • Bit-Width Floor: While 4-bit quantization (W4/U4) is highly effective, the paper does not explore ultra-low bit-widths (e.g., 2-bit or binary), which may require even more aggressive calibration or quantization-aware training (QAT).
  • Calibration Time: Although much faster than full retraining, the calibration process still takes between 5 and 20 minutes on an NVIDIA RTX 4090, depending on the model size.
  • Architecture Specificity: The current analysis is tailored to Leaky Integrate-and-Fire (LIF) neurons. Extending this to more complex biological neuron models or different spiking architectures remains an open area for research.

Summary updated

Abstract

Spiking transformers combine attention-based representation learning with sparse spike communication, yet their weights and membrane variables still incur substantial storage and computation overhead. Post-training quantization (PTQ) offers a practical way to reduce these overheads, but preserving neuronal firing under weight and membrane quantization remains challenging. In this paper, we propose Decision-Aligned Post-Training Quantization (DAQ), a method that calibrates the quantizers to align neuronal firing decisions with those of a floating-point reference. We analyze how errors from these quantizers affect firing at decision boundaries and how they propagate over time steps in recurrent neurons. Based on this analysis, we formulate an objective that encourages quantized membrane variables to remain within the reference firing regions along the quantized model's own state trajectory. Experiments across multiple spiking transformer architectures on image classification, semantic segmentation, and event-based tracking demonstrate the effectiveness of DAQ with 4-bit weights and membrane variables. On FE108, DAQ achieves 58.8\% AUC and 90.9\% PR with SDTrack, while reducing energy by 66.2\%.
BibTeX
@misc{chen2026daq,
  author = {Jialin Chen and Xiaochen Zhao and Aili Wang},
  title = {{DAQ: Decision-Aligned Post-Training Quantization for Spiking Transformers}},
  year = {2026},
  eprint = {2611.22611},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22611},
}
Figure