Generative Particle Variational Inference via Estimation of Functional Gradients

Neale Ratzlaff, Qinxun Bai, Fuxin Li, Wei Xu

TL;DR

This paper introduces Generative Particle Variational Inference (GPVI), a novel method for Bayesian inference that combines the strengths of particle-based methods (avoiding parametric assumptions) with generative models (flexible sampling). GPVI learns a neural sampler to directly generate particles from a target posterior distribution by estimating the functional gradient of the KL-divergence, outperforming existing generative approaches and rivaling traditional methods like HMC.

What Problem Does This Paper Address?

Bayesian inference is powerful for reasoning under uncertainty, but computing the posterior distribution is often intractable, especially for complex models like deep neural networks. Traditional methods like Markov Chain Monte Carlo (MCMC) can be slow, while standard Variational Inference (VI) often relies on restrictive parametric assumptions about the posterior. Particle-based VI (ParVI) offers a non-parametric alternative but typically requires a fixed set of particles, limiting its ability to generate new samples. Amortized ParVI methods address this by training a neural sampler, but they can suffer from suboptimality due to how gradients are backpropagated. This paper aims to bridge this gap by developing a generative ParVI method that offers both the asymptotic accuracy of ParVI and the sampling flexibility of generative models, without the suboptimality of existing amortized approaches.

Methods

The core innovation is Generative Particle Variational Inference (GPVI). Instead of directly optimizing a neural sampler's parameters, GPVI estimates the functional gradient of the KL-divergence between the distribution of generated particles and the target posterior. This functional gradient is then "pulled back" to update the neural sampler's parameters.

Key technical aspects include:

  • Functional Gradient Estimation: The paper derives an explicit formula for the functional gradient, assuming it lies within a Reproducing Kernel Hilbert Space (RKHS).
  • Helper Network for Jacobian Inverse: A major computational hurdle is efficiently computing the inverse of the neural sampler's Jacobian. GPVI introduces a helper network that learns to approximate the required inverse Jacobian-vector product. This avoids direct Jacobian inversion, which is computationally expensive and numerically unstable for high-dimensional outputs.
  • Parameterization of the Generator: To ensure the Jacobian is square and the function is injective (necessary for change of variables), a specific parameterization for the generator network is proposed, using a "slimmer" network for the core transformation.
  • Alternating Training: The generator and helper networks are trained iteratively using stochastic gradient descent.

Datasets

The paper evaluates GPVI on several standard benchmarks:

  • Density Estimation: 2D and 5D zero-mean Gaussian distributions with non-diagonal covariances.
  • Bayesian Linear Regression: A synthetic dataset where the true posterior is analytically known.
  • Multimodal Classification: A 2D, 4-class classification problem with a mixture of Gaussian distributions.
  • Image Classification (Open Category): MNIST and CIFAR-10 datasets, where the task is to detect outlier classes not seen during training.

No new datasets were introduced.

Key Insights and Ideas

  • Functional Gradient is Key: Directly optimizing the functional gradient of the KL-divergence, rather than an amortized version, is crucial for maintaining the desirable properties of ParVI methods while enabling generative sampling.
  • Helper Network for Practicality: The introduction of a helper network to approximate the inverse Jacobian-vector product is a critical enabler for GPVI's scalability and practical application, especially in high-dimensional settings. This bypasses the need for computationally prohibitive direct Jacobian inversion.
  • Amortized SVGD's Suboptimality: The paper provides theoretical and empirical evidence that amortized SVGD's approach of backpropagating the SVGD gradient can lead to suboptimality, often resulting in underestimation of uncertainty, especially when the generator's Jacobian is not the identity.
  • GPVI's Robustness: GPVI demonstrates strong performance across various tasks, from fitting known distributions to handling complex, intractable posteriors in Bayesian Neural Networks (BNNs), consistently outperforming amortized generative methods.

Key Results

  • Density Estimation: GPVI achieved competitive performance, matching or exceeding other generative ParVI methods and performing well on both 2D and 5D Gaussian distributions.
  • Bayesian Linear Regression: GPVI outperformed amortized ParVI, MF-VI, and even HMC in terms of fitting the learned mean and covariance parameters, demonstrating its ability to accurately approximate known posteriors.
  • 4-Class Classification: GPVI showed better uncertainty estimation than amortized SVGD and MF-VI, closely matching SVGD and outperforming HMC in regions far from training data.
  • Open Category Classification (MNIST/CIFAR-10): GPVI achieved high AUC and competitive ECE scores, demonstrating its effectiveness in detecting outlier inputs and estimating uncertainty in large-scale image classification tasks. It consistently outperformed amortized ParVI methods.
  • Computational Efficiency: While requiring a helper network, GPVI's training time overhead compared to amortized SVGD was found to be acceptable (e.g., 0.28x for MNIST, 0.78x for CIFAR-10).

Limitations and Future Work

  • Computational Cost: While GPVI is more efficient than traditional MCMC for sampling, training the generator and helper network can still be computationally intensive, especially for very large models or datasets. The paper notes that the helper network's complexity scales with the generator's output dimension.
  • Hyperparameter Tuning: Like most deep learning methods, GPVI relies on careful hyperparameter tuning.
  • Future Work: The authors suggest exploring GPVI's application to large-scale tasks like image generation, which would further test its scalability and generative capabilities. They also mention the potential for using GPVI in scenarios where the true posterior is unknown, relying on its ability to approximate complex distributions.

Summary updated

Abstract

Recently, particle-based variational inference (ParVI) methods have gained interest because they can avoid arbitrary parametric assumptions that are common in variational inference. However, many ParVI approaches do not allow arbitrary sampling from the posterior, and the few that do allow such sampling suffer from suboptimality. This work proposes a new method for learning to approximately sample from the posterior distribution. We construct a neural sampler that is trained with the functional gradient of the KL-divergence between the empirical sampling distribution and the target distribution, assuming the gradient resides within a reproducing kernel Hilbert space. Our generative ParVI (GPVI) approach maintains the asymptotic performance of ParVI methods while offering the flexibility of a generative sampler. Through carefully constructed experiments, we show that GPVI outperforms previous generative ParVI methods such as amortized SVGD, and is competitive with ParVI as well as gold-standard approaches like Hamiltonian Monte Carlo for fitting both exactly known and intractable target distributions.
BibTeX
@inproceedings{ratzlaff2021generative,
  author = {Neale Ratzlaff and Qinxun Bai and Fuxin Li and Wei Xu},
  title = {{Generative Particle Variational Inference via Estimation of Functional Gradients}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
  url = {https://openreview.net/forum?id=HRzWMw2Rvx9},
}
Figure