$f$-Divergence Thermodynamic Variational Objective: a Deformed Geometry Perspective
Lists
Only you see this
Abstract
In this paper, we propose a $f$-divergence Thermodynamic Variational Objective ($f$-TVO). $f$-TVO generalizes the Thermodynamic Variational Objective (TVO) by replacing Kullback–Leibler (KL) divergence with arbitary differeitiable $f$-divergence. In particular, $f$-TVO approximates dual function of model evidence $f^*(p(x))$ rather than the log model evidence $\log p(x)$ in TVO. $f$-TVO is derived from a deformed $\chi$-geometry perspective. By defining $\chi$-exponential family exponential, we are able to integral $f$-TVO along the $\chi$-path, which is the deformed geodesic between variational posterior distribution and true posterior distribution. Optimizing scheme of $f$-TVO includes reparameterization trick and Monte Carlo approximation. Experiments on VAE and Bayesian neural network show that the proposed $f$-TVO performs better than cooresponding baseline $f$-divergence variational inference.
BibTeX
@inproceedings{li2022f,
author = {Jun Li and Ping Li},
title = {{$f$-Divergence Thermodynamic Variational Objective: a Deformed Geometry Perspective}},
booktitle = {International Conference on Learning Representations},
year = {2022},
url = {https://openreview.net/forum?id=mhv2gWm3sf},
}Loading related content...