The Adversarial Vulnerability of Bayesian Inference

52 concepts29 papers1 proposal

In machine learning, Bayesian methods are often viewed as the ultimate tool for reliability. Instead of relying on a single "best-guess" point estimate for model weights, a Bayesian Neural Network (BNN) or a Gaussian Process (GP) maintains a probability distribution over all possible weights given the training data. When making a prediction, the model integrates over this entire posterior distribution:

$$p(y|x) = \int p(y|x, w)p(w|\mathcal{D})dw$$

Intuitively, this integration acts like an infinite ensemble. If an adversary crafts a tiny, malicious perturbation $\delta$ designed to exploit a specific blind spot in one set of weights, averaging predictions across a whole distribution of weights should theoretically wash out that vulnerability. Early work in this space, such as that by Ritter et al. and Doan et al., leaned heavily into this intuition, suggesting that the ensemble-like behavior of BNNs inherently smooths out decision boundaries and mitigates high-dimensional vulnerabilities.

However, a growing body of research shows that this "probabilistic shield" is largely an illusion. Bayesian inference is deeply vulnerable to adversarial manipulation, and understanding why requires looking closely at the geometry of high-dimensional spaces and the mechanics of posterior approximations.

The Reality Check: Deconstructing the Illusion

When subjected to rigorous, corrected evaluation protocols, standard Bayesian models do not inherently resist adversarial attacks. Feng et al. demonstrated that when implementation errors from earlier studies are corrected, BNNs—even those using "gold-standard" exact inference methods—are just as vulnerable to being fooled as their deterministic counterparts. Similarly, Mitros et al. showed that approximate Bayesian inference methods can still see their accuracy drop to near zero under targeted noise.

This raises a fundamental question: Is this vulnerability a failure of our approximations of the posterior, or is it an intrinsic property of the neural network architecture itself?

Evidence points toward the architecture. The geometric instability that allows adversarial examples to exist—where classification boundaries are tightly packed and close to data points in high-dimensional space—is a structural property of the network. Even if you sample weights from a perfect posterior distribution, the underlying functions still share these geometric vulnerabilities.

How Posterior Approximations Alter Vulnerability

While the underlying architecture creates the vulnerability, the method we use to approximate the intractable Bayesian posterior dictates how gracefully the model fails. We cannot compute the true posterior for deep networks, so we rely on approximations. As Cardelli et al. discovered, different posterior approximation methods yield drastically different robustness profiles. Some approximations cause a BNN to behave almost deterministically, completely failing to signal uncertainty when an input is adversarially manipulated.

A key factor here is the distinction between local and global posterior approximations:

  • Local Approximations (e.g., Laplace Approximation): The Laplace method fits a Gaussian distribution locally around a single mode (a peak) of the loss landscape. Because it only captures the geometry of that immediate neighborhood, its uncertainty estimates can become highly unreliable when an adversary pushes an input into a region where the local approximation no longer holds.
  • Global Approximations (e.g., Expectation Propagation): As shown by Patane et al., global approximations do a much better job of stabilizing decision boundaries. By optimizing for a consistent representation of uncertainty across the entire input domain rather than around a single point, global methods prevent the model's uncertainty metrics from being easily bypassed by small input perturbations.

Weaponizing the Posterior

Remarkably, the fact that Bayesian models reason about distributions rather than point estimates can actually be weaponized by an adversary to create stronger attacks.

Consider a black-box transfer attack, where an adversary generates an adversarial example on a "substitute" model hoping it will fool a hidden "victim" model. If the adversary uses a standard deterministic model as the substitute, the generated attack often overfits to the highly specific decision boundaries of that exact model, failing to transfer to the victim.

Li demonstrated that making the substitute model Bayesian dramatically enhances the transferability of adversarial examples. By optimizing the perturbation against a distribution of weights rather than a single point estimate, the attack is forced to find a vulnerability that is robust across an entire space of functions. This effectively simulates an infinite ensemble, preventing the attack from overfitting to a single model configuration and uncovering universal geometric blind spots that easily fool the victim model.

Deterministic Substitute:
Input x ───> [ Single Weight Set w ] ───> Overfits to specific boundary

Bayesian Substitute:
Input x ───> [ Weight Distribution p(w) ] ───> Finds universal vulnerability
                                               (Highly Transferable)

Vulnerabilities also extend back to the training phase. Carreau et al. showed that Bayesian inference is highly susceptible to "surgical" data poisoning. Because Bayesian updates follow strict mathematical rules based on the likelihood function, an attacker can use stochastic integer programming to calculate exactly which data points to delete or replicate to systematically steer the posterior distribution toward a malicious target belief, all while leaving the rest of the model seemingly undisturbed.

Engineering True Certifiable Robustness

Because empirical uncertainty estimation does not equal security, researchers have shifted toward building Bayesian frameworks with mathematical guarantees.

One approach is Bayesian Adversarial Learning (BAL). Introduced by Ye et al., BAL moves away from the traditional frequentist min-max optimization (which trains on the single worst-case perturbation). Instead, it treats both the model parameters and the adversarial perturbation generation as probability distributions. By accounting for the uncertainty in how adversarial data is generated, it creates a broader defensive shield that generalizes better to unseen attack types.

For safety-critical applications, we need formal verification. To achieve this, Wicker et al. integrated Interval Bound Propagation (IBP) directly into the Bayesian likelihood function. IBP computes the upper and lower bounds of a network's outputs given an input bounding box. By forcing the Bayesian update to maximize the likelihood of the worst-case bound within a perturbation radius, the authors created BNNs that are certifiably robust.

To scale these guarantees, Adams et al. framed BNNs as stochastic dynamical systems. By applying dynamic programming combined with convex relaxations, they bypassed the need to sample millions of weights to verify safety, producing formal verification bounds that are orders of magnitude tighter and faster to compute.

Ultimately, the intersection of adversarial machine learning and Bayesian inference teaches us a vital lesson: modeling uncertainty is not a shortcut to security. True robustness requires explicitly designing the training objective or the posterior approximation to withstand adversarial geometry.

Key Foundational Work

Bayesian Adversarial Learning (2018) — contributed 2 concept elements to this direction.
A Scalable Laplace Approximation for Neural Networks (2018) — contributed 1 concept elements to this direction.
Asymptotic Analysis of Distributed Bayesian Detection with Byzantine Data (2014) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_adversarial_vulnerability_of_bayesian_in_ce732760,
  title = {adversarial vulnerability of Bayesian inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adversarial-vulnerability-of-bayesian-inference/rdr_ce73276010c74c12907aff7afac36e2b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_adversarial_vulnerability_of_bayesian_in_ce732760_v1,
  title = {adversarial vulnerability of Bayesian inference},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adversarial-vulnerability-of-bayesian-inference/rdr_ce73276010c74c12907aff7afac36e2b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake