Introduction
Hyperdimensional (HD) computing has emerged as a promising brain-inspired paradigm for machine learning, particularly suited for Internet of Things (IoT) and edge devices. Unlike traditional neural networks that rely on complex matrix calculus, HD computing operates on massive vectors—often exceeding 10,000 dimensions—using simple bitwise operations. This inherent simplicity offers significant advantages in terms of hardware efficiency and energy consumption.
The Binarization Tax in HD Computing
Despite its potential, HD computing faces a significant hurdle known as the "binarization tax." To maximize energy efficiency on hardware platforms like FPGAs, high-precision vectors must be converted into binary form (values restricted to -1 or 1). Standard deterministic binarization, typically using a sign function, results in substantial information loss, leading to a marked decrease in classification accuracy.
While previous methods like the QuantHD algorithm attempted to mitigate this through iterative retraining, they often suffer from slow convergence and instability. These deterministic approaches frequently get trapped in "local minima," where the model ceases to improve despite further training iterations.
The QubitHD Approach: Stochastic Binarization
To address these limitations, the authors introduce QubitHD, a novel method that replaces deterministic binarization with a stochastic process. Inspired by the behavior of quantum bits (qubits), where a state exists in superposition until measurement, QubitHD treats high-precision values as "quantum states" and binarization as a probabilistic "measurement."
As illustrated in Figure 4, while QuantHD uses a hard threshold at zero, QubitHD introduces a "cutoff" range $[-b, b]$. Values falling within this range are binarized stochastically: the probability of a value becoming +1 is proportional to its magnitude. This ensures that the expected value of the binarized vector remains equal to the original high-precision value, effectively preserving the statistical "essence" of the data.
This stochasticity serves a dual purpose. First, it prevents the model from getting stuck in local minima by introducing controlled noise during training. Second, it enables the use of simple Hamming Distance (XOR followed by bit counting) for similarity calculations on FPGAs, replacing more computationally expensive operations.
Experimental Evaluation
The performance of QubitHD was evaluated across five diverse classification benchmarks:
- ISOLET: Voice recognition (26 classes).
- UCIHAR: Human activity recognition via mobile sensors.
- MNIST: Handwritten digit recognition.
- FACE: Face recognition.
- EXTRA: Phone position recognition.
Results and Discussion
QubitHD demonstrates superior performance in both accuracy and training efficiency. On average, the method closed the accuracy gap between binarized and non-binarized HD models by 38.8%.
As shown in Figure 1, QubitHD (variable-threshold) achieves higher maximum accuracy and converges significantly faster than the baseline binary model and fixed-threshold variants. Specifically, it reaches optimal accuracy 30–50% faster than QuantHD.
Hardware implementation on a Kintex-7 FPGA further highlights the efficiency of the approach. QubitHD provided a 65% improvement in energy efficiency and a 95% improvement in training time compared to state-of-the-art HD algorithms. Furthermore, when compared to Binarized Neural Networks (BNNs), QubitHD was 52x faster during inference and an order of magnitude more energy-efficient during training, while maintaining competitive accuracy levels.
Conclusion
QubitHD successfully bridges the gap between high-precision accuracy and binary hardware efficiency in HD computing. By leveraging stochastic binarization, the algorithm provides a more stable and faster training process that is highly optimized for edge deployment. Future research may focus on automating the tuning of the stochastic threshold $b$ and extending this approach to non-supervised tasks such as clustering.
Summary updated