A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel

Yindong Chen, Yiwei Wang, Lulu Kang, Chun Liu

Abstract

We propose a novel deterministic sampling method, EVI-MMD, to approximate a target distribution $ρ^*$ by minimizing the kernel discrepancy, also known as the Maximum Mean Discrepancy (MMD). Leveraging the energetic variational inference framework (Wang et al., 2021), we transform the MMD minimization problem into solving a dynamic system of Ordinary Differential Equations (ODEs) for particles. The implicit Euler scheme is employed to solve the ODE system, leading to a proximal minimization problem at each iteration, which is efficiently addressed using optimization algorithms such as L-BFGS. A key innovation of our method is a dynamic bandwidth selection strategy for the Gaussian kernel, which, although heuristic at this stage, represents a meaningful step toward addressing a long-standing challenge in kernel-based methods. Comprehensive numerical experiments demonstrate that this adaptive bandwidth significantly enhances the performance of EVI-MMD. We apply the EVI-MMD algorithm to two types of sampling problems: (1) when the target distribution is fully specified by a density function, and (2) the ``two-sample problem,'' where only training data are available. In the latter case, EVI-MMD serves as a generative model, producing new samples that faithfully replicate the distribution of the training data. With carefully tuned parameters, EVI-MMD outperforms several existing methods in both scenarios.
BibTeX
@misc{chen2021deterministic,
  author = {Yindong Chen and Yiwei Wang and Lulu Kang and Chun Liu},
  title = {{A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel}},
  year = {2021},
  eprint = {2111.10722},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2111.10722},
}

Loading related content...

Figure