QubitHD: A Stochastic Acceleration Method for HD Computing-Based Machine Learning

Samuel Bosch, Alexander Sanchez de la Cerda, Mohsen Imani, Tajana Simunic Rosing, Giovanni De MicheliarXiv 2019 · arXiv

Introduction

Hyperdimensional (HD) computing has emerged as a promising brain-inspired paradigm for machine learning, particularly suited for Internet of Things (IoT) and edge devices. Unlike traditional neural networks that rely on complex matrix calculus, HD computing operates on massive vectors—often exceeding 10,000 dimensions—using simple bitwise operations. This inherent simplicity offers significant advantages in terms of hardware efficiency and energy consumption.

The Binarization Tax in HD Computing

Despite its potential, HD computing faces a significant hurdle known as the "binarization tax." To maximize energy efficiency on hardware platforms like FPGAs, high-precision vectors must be converted into binary form (values restricted to -1 or 1). Standard deterministic binarization, typically using a sign function, results in substantial information loss, leading to a marked decrease in classification accuracy.

While previous methods like the QuantHD algorithm attempted to mitigate this through iterative retraining, they often suffer from slow convergence and instability. These deterministic approaches frequently get trapped in "local minima," where the model ceases to improve despite further training iterations.

The QubitHD Approach: Stochastic Binarization

To address these limitations, the authors introduce QubitHD, a novel method that replaces deterministic binarization with a stochastic process. Inspired by the behavior of quantum bits (qubits), where a state exists in superposition until measurement, QubitHD treats high-precision values as "quantum states" and binarization as a probabilistic "measurement."

Comparison of deterministic QuantHD binarization vs. stochastic QubitHD binarization.

As illustrated in Figure 4, while QuantHD uses a hard threshold at zero, QubitHD introduces a "cutoff" range $[-b, b]$. Values falling within this range are binarized stochastically: the probability of a value becoming +1 is proportional to its magnitude. This ensures that the expected value of the binarized vector remains equal to the original high-precision value, effectively preserving the statistical "essence" of the data.

This stochasticity serves a dual purpose. First, it prevents the model from getting stuck in local minima by introducing controlled noise during training. Second, it enables the use of simple Hamming Distance (XOR followed by bit counting) for similarity calculations on FPGAs, replacing more computationally expensive operations.

Experimental Evaluation

The performance of QubitHD was evaluated across five diverse classification benchmarks:

  • ISOLET: Voice recognition (26 classes).
  • UCIHAR: Human activity recognition via mobile sensors.
  • MNIST: Handwritten digit recognition.
  • FACE: Face recognition.
  • EXTRA: Phone position recognition.

Results and Discussion

QubitHD demonstrates superior performance in both accuracy and training efficiency. On average, the method closed the accuracy gap between binarized and non-binarized HD models by 38.8%.

Accuracy improvement and convergence speed of QubitHD compared to baseline models.

As shown in Figure 1, QubitHD (variable-threshold) achieves higher maximum accuracy and converges significantly faster than the baseline binary model and fixed-threshold variants. Specifically, it reaches optimal accuracy 30–50% faster than QuantHD.

Hardware implementation on a Kintex-7 FPGA further highlights the efficiency of the approach. QubitHD provided a 65% improvement in energy efficiency and a 95% improvement in training time compared to state-of-the-art HD algorithms. Furthermore, when compared to Binarized Neural Networks (BNNs), QubitHD was 52x faster during inference and an order of magnitude more energy-efficient during training, while maintaining competitive accuracy levels.

Conclusion

QubitHD successfully bridges the gap between high-precision accuracy and binary hardware efficiency in HD computing. By leveraging stochastic binarization, the algorithm provides a more stable and faster training process that is highly optimized for edge deployment. Future research may focus on automating the tuning of the stochastic threshold $b$ and extending this approach to non-supervised tasks such as clustering.

Summary updated

Abstract

Machine Learning algorithms based on Brain-inspired Hyperdimensional(HD) computing imitate cognition by exploiting statistical properties of high-dimensional vector spaces. It is a promising solution for achieving high energy efficiency in different machine learning tasks, such as classification, semi-supervised learning, and clustering. A weakness of existing HD computing-based ML algorithms is the fact that they have to be binarized to achieve very high energy efficiency. At the same time, binarized models reach lower classification accuracies. To solve the problem of the trade-off between energy efficiency and classification accuracy, we propose the QubitHD algorithm. It stochastically binarizes HD-based algorithms, while maintaining comparable classification accuracies to their non-binarized counterparts. The FPGA implementation of QubitHD provides a 65% improvement in terms of energy efficiency, and a 95% improvement in terms of training time, as compared to state-of-the-art HD-based ML algorithms. It also outperforms state-of-the-art low-cost classifiers (such as Binarized Neural Networks) in terms of speed and energy efficiency by an order of magnitude during training and inference.
BibTeX
@misc{bosch2019qubithd,
  author = {Samuel Bosch and Alexander Sanchez de la Cerda and Mohsen Imani and Tajana Simunic Rosing and Giovanni De Micheli},
  title = {{QubitHD: A Stochastic Acceleration Method for HD Computing-Based Machine Learning}},
  year = {2019},
  eprint = {1911.12446},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1911.12446},
}
Figure