Generative Particle Variational Inference via Estimation of Functional Gradients

Neale Ratzlaff, Qinxun Bai, Fuxin Li, Wei Xu

This paper introduces Generative Particle Variational Inference (GPVI), a method that combines the flexibility of neural samplers with the asymptotic accuracy of particle-based variational inference.

The Challenge of Bayesian Inference

Bayesian inference is the "gold standard" for reasoning under uncertainty, but calculating the posterior distribution for complex models like Deep Neural Networks (BNNs) is notoriously difficult.

Two common approaches exist:

  1. Variational Inference (VI): Usually assumes a simple parametric form (like a Gaussian) for the posterior. This is often too restrictive to capture complex, multimodal distributions.
  2. Particle-based Variational Inference (ParVI): Methods like Stein Variational Gradient Descent (SVGD) represent the posterior using a set of "particles" that are iteratively pushed toward the target. While accurate, these are limited to a fixed set of particles; you cannot easily "sample" a new one after training.

Existing "amortized" versions of ParVI try to train a neural network (a sampler) to produce these particles. However, these amortized methods often fail to converge as well as the original particle methods because they do not follow the "steepest descent" direction in the space of functions.

Generative Particle Variational Inference (GPVI)

The authors propose GPVI, which trains a neural sampler by directly estimating the functional gradient of the KL-divergence between the sampler's distribution and the target posterior.

Core Innovations

  • Functional Pullback: Instead of naively backpropagating gradients, GPVI calculates the gradient in a Reproducing Kernel Hilbert Space (RKHS) and "pulls it back" to the sampler's parameters. This ensures the sampler follows the optimal descent path.
  • The Helper Network: The functional gradient formula requires the inverse of the sampler's input-output Jacobian. For a BNN with millions of parameters, calculating this inverse is computationally impossible ($O(d^3)$). To solve this, the authors introduce a small helper network. This network is trained via a simple regression loss to estimate the product of the inverse Jacobian and a vector, distributing the heavy computational cost over the training process.
  • Injective Parameterization: To ensure the math for probability density changes holds up, they use a specific architecture for the generator that guarantees the mapping is injective (one-to-one) and makes the Jacobian easier to handle.

Key Insights

  • The Amortization Gap: The paper explains why previous amortized methods (like Amortized SVGD) underperform: they use a non-steepest descent direction that becomes less optimal as the sampler updates.
  • Efficiency via Estimation: The helper network is a clever workaround for the Jacobian inverse. By treating the inverse Jacobian-vector product as a learning problem rather than a matrix inversion problem, they make functional gradients viable for high-dimensional neural networks.
  • Better Uncertainty: GPVI captures "epistemic uncertainty" (uncertainty due to lack of data) much better than standard Variational Inference or Deep Ensembles, which tend to be overconfident in regions without training data.

Results

The authors evaluated GPVI across several benchmarks, including synthetic distributions, Bayesian linear regression, and image classification.

  • Density Estimation: In 5D Gaussian tests, GPVI achieved a variance error of 0.14, significantly lower than Amortized SVGD (0.37) and Amortized KSD (1.68).
  • Open-Category (CIFAR-10): GPVI achieved an Expected Calibration Error (ECE) of 0.018, outperforming standard SVGD (0.055) and Amortized SVGD (0.073). This indicates that GPVI is much better at "knowing what it doesn't know."
  • Qualitative Success: In regression tasks with gaps in training data, GPVI successfully places high uncertainty in those gaps, closely matching the gold-standard Hamiltonian Monte Carlo (HMC), whereas other amortized methods tend to be overconfident or poorly fitted.
Comparison of uncertainty estimation across methods

Limitations and Future Work

  • Helper Network Scaling: While the helper network avoids $O(d^3)$ costs, its size still scales with the output dimension of the generator. For extremely large models, this could still become a bottleneck.
  • Hyperparameters: The method introduces new hyperparameters, such as the noise dimension and the regularization constant, which may require tuning for different tasks.
  • Future Directions: The authors suggest exploring GPVI for large-scale generative tasks, such as high-resolution image generation, where capturing the full diversity of a distribution is critical.

Summary updated

Abstract

Recently, particle-based variational inference (ParVI) methods have gained interest because they can avoid arbitrary parametric assumptions that are common in variational inference. However, many P...
BibTeX
@inproceedings{ratzlaff2021generative,
  author = {Neale Ratzlaff and Qinxun Bai and Fuxin Li and Wei Xu},
  title = {{Generative Particle Variational Inference via Estimation of Functional Gradients}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=HRzWMw2Rvx9},
}
Figure