On the effects of quantisation on model uncertainty in bayesian neural networks

Martin Ferianc, Partha Maji, Matthew Mattina, Miguel R. D. Rodrigues

Introduction

Bayesian Neural Networks (BNNs) are widely regarded as the gold standard for uncertainty estimation in deep learning. However, their high computational cost often precludes deployment in real-world, resource-constrained environments. This research investigates whether BNNs can be compressed using Quantisation Aware Training (QAT) to 8-bit integer representations without compromising their predictive accuracy or uncertainty calibration, making them viable for safety-critical applications on edge devices.

The Overconfidence Problem and the Bayesian Solution

Standard "pointwise" neural networks are prone to overconfidence, frequently assigning high probability to incorrect predictions or out-of-distribution data. BNNs mitigate this by treating weights as probability distributions, enabling well-calibrated uncertainty estimates. Despite these benefits, BNNs incur a "Bayesian tax"—the requirement for multiple Monte Carlo (MC) sampling passes, which increases memory and compute demands.

While quantisation (converting 32-bit floats to 8-bit integers) is a standard technique for optimizing pointwise networks for edge devices, its impact on the delicate uncertainty information in BNNs was previously unclear. The authors investigate whether we can enjoy hardware benefits without losing the safety benefits of Bayesian inference.

Methodology: Quantising Bayesian Inference

The authors applied Uniform Affine Quantisation to three prominent Bayesian inference schemes: Monte Carlo Dropout (MCD), Bayes-By-Backprop (BBB), and Stochastic Gradient Langevin Dynamics with Hamiltonian Monte Carlo (SGHMC).

To bridge the gap between floating-point training and integer deployment, they employed Quantisation Aware Training (QAT). This process utilizes Simulated Quantisation (SQ) blocks during training to model rounding errors, allowing the network to adapt to low-bit precision. Crucially, both weights and activations were quantised to ensure compatibility with modern hardware accelerators.

Quantized Bayes-By-Backprop (BBB) layer structure.

As shown in the figure above for the BBB method, SQ blocks are integrated into the parameter sampling process, ensuring that the learned distributions (defined by $\mu$ and $\sigma$) are robust to eventual discretisation.

Experimental Evaluation

The methodology was rigorously tested across several benchmarks:

  • Regression: Six UCI datasets (e.g., Housing, Concrete).
  • Image Classification: MNIST and CIFAR-10.
  • Out-of-Distribution (OOD) Detection: Testing models on FashionMNIST (trained on MNIST) and SVHN (trained on CIFAR-10) to see if they recognize "unknown" data.
  • Robustness: Evaluating performance under domain shifts like image rotations and brightness changes.

Key Results and Insights

Bayesian Robustness to Quantisation

A primary insight is that BNNs exhibit greater resilience to quantisation than pointwise networks. While standard networks see rapid performance degradation as bit-width decreases, BNNs maintain accuracy and calibration for much longer. The authors suggest that the noise introduced by quantisation may actually act as a form of regularisation.

Preservation of Uncertainty

Quantised BNNs continue to provide reliable uncertainty signals. Even at 8-bit precision, they maintain high predictive entropy when encountering OOD data, avoiding the false confidence typical of compressed pointwise models.

NLL across varying activation bit-widths (fixed 8-bit weights).

As illustrated in the Negative Log Likelihood (NLL) results above, BNN variants (MCD, BBB, SGHMC) consistently outperform the pointwise baseline in terms of calibration across various bit-widths. While the pointwise model's NLL is high and unstable, BNNs maintain low, stable NLL values until extreme compression (below 4 bits) is reached.

The Activation Bottleneck

The study identified that quantising activations is generally more detrimental than quantising weights. Most models experienced performance collapse when activations were reduced below 4 bits, whereas weights were more robust to compression.

Conclusion

The research concludes that 8-bit (W8A8) quantisation is a "sweet spot," offering a 4x reduction in memory footprint and significant latency gains with negligible loss in uncertainty quality. While SGHMC proved slightly more sensitive due to its reliance on ensemble diversity, the overall findings suggest that Bayesian safety and hardware efficiency are not mutually exclusive. Future work may explore mixed-precision BNNs to push compression limits even further.

Summary updated

Abstract

Bayesian neural networks (BNNs) are making significant progress in many research areas where decision-making needs to be accompanied by uncertainty estimation. Being able to quantify uncertainty while making decisions is essential for understanding when the model is over-/under-confident, and hence BNNs are attracting interest in safety-critical applications, such as autonomous driving, healthcare, and robotics. Nevertheless, BNNs have not been as widely used in industrial practice, mainly because of their increased memory and compute costs. In this work, we investigate quantisation of BNNs by compressing 32-bit floating-point weights and activations to their integer counterparts, that has already been successful in reducing the compute demand in standard pointwise neural networks. We study three types of quantised BNNs, we evaluate them under a wide range of different settings, and we empirically demonstrate that a uniform quantisation scheme applied to BNNs does not substantially decrease their quality of uncertainty estimation.
BibTeX
@inproceedings{ferianc2021effects,
  author = {Martin Ferianc and Partha Maji and Matthew Mattina and Miguel R. D. Rodrigues},
  title = {{On the effects of quantisation on model uncertainty in bayesian neural networks}},
  booktitle = {Conference on Uncertainty in Artificial Intelligence},
  year = {2021},
}
Figure