Introduction
Bayesian Neural Networks (BNNs) are widely regarded as the gold standard for uncertainty estimation in deep learning. However, their high computational cost often precludes deployment in real-world, resource-constrained environments. This research investigates whether BNNs can be compressed using Quantisation Aware Training (QAT) to 8-bit integer representations without compromising their predictive accuracy or uncertainty calibration, making them viable for safety-critical applications on edge devices.
The Overconfidence Problem and the Bayesian Solution
Standard "pointwise" neural networks are prone to overconfidence, frequently assigning high probability to incorrect predictions or out-of-distribution data. BNNs mitigate this by treating weights as probability distributions, enabling well-calibrated uncertainty estimates. Despite these benefits, BNNs incur a "Bayesian tax"—the requirement for multiple Monte Carlo (MC) sampling passes, which increases memory and compute demands.
While quantisation (converting 32-bit floats to 8-bit integers) is a standard technique for optimizing pointwise networks for edge devices, its impact on the delicate uncertainty information in BNNs was previously unclear. The authors investigate whether we can enjoy hardware benefits without losing the safety benefits of Bayesian inference.
Methodology: Quantising Bayesian Inference
The authors applied Uniform Affine Quantisation to three prominent Bayesian inference schemes: Monte Carlo Dropout (MCD), Bayes-By-Backprop (BBB), and Stochastic Gradient Langevin Dynamics with Hamiltonian Monte Carlo (SGHMC).
To bridge the gap between floating-point training and integer deployment, they employed Quantisation Aware Training (QAT). This process utilizes Simulated Quantisation (SQ) blocks during training to model rounding errors, allowing the network to adapt to low-bit precision. Crucially, both weights and activations were quantised to ensure compatibility with modern hardware accelerators.
As shown in the figure above for the BBB method, SQ blocks are integrated into the parameter sampling process, ensuring that the learned distributions (defined by $\mu$ and $\sigma$) are robust to eventual discretisation.
Experimental Evaluation
The methodology was rigorously tested across several benchmarks:
- Regression: Six UCI datasets (e.g., Housing, Concrete).
- Image Classification: MNIST and CIFAR-10.
- Out-of-Distribution (OOD) Detection: Testing models on FashionMNIST (trained on MNIST) and SVHN (trained on CIFAR-10) to see if they recognize "unknown" data.
- Robustness: Evaluating performance under domain shifts like image rotations and brightness changes.
Key Results and Insights
Bayesian Robustness to Quantisation
A primary insight is that BNNs exhibit greater resilience to quantisation than pointwise networks. While standard networks see rapid performance degradation as bit-width decreases, BNNs maintain accuracy and calibration for much longer. The authors suggest that the noise introduced by quantisation may actually act as a form of regularisation.
Preservation of Uncertainty
Quantised BNNs continue to provide reliable uncertainty signals. Even at 8-bit precision, they maintain high predictive entropy when encountering OOD data, avoiding the false confidence typical of compressed pointwise models.
As illustrated in the Negative Log Likelihood (NLL) results above, BNN variants (MCD, BBB, SGHMC) consistently outperform the pointwise baseline in terms of calibration across various bit-widths. While the pointwise model's NLL is high and unstable, BNNs maintain low, stable NLL values until extreme compression (below 4 bits) is reached.
The Activation Bottleneck
The study identified that quantising activations is generally more detrimental than quantising weights. Most models experienced performance collapse when activations were reduced below 4 bits, whereas weights were more robust to compression.
Conclusion
The research concludes that 8-bit (W8A8) quantisation is a "sweet spot," offering a 4x reduction in memory footprint and significant latency gains with negligible loss in uncertainty quality. While SGHMC proved slightly more sensitive due to its reliance on ensemble diversity, the overall findings suggest that Bayesian safety and hardware efficiency are not mutually exclusive. Future work may explore mixed-precision BNNs to push compression limits even further.
Summary updated