In modern machine learning, we frequently encounter probability distributions that are "unnormalized." Whether it is a posterior distribution in Bayesian inference or an energy-based model in deep learning, we often know the density function only up to an intractable constant $Z$ (the partition function). This makes traditional tools like the Kullback-Leibler (KL) divergence or the Maximum Mean Discrepancy (MMD) difficult to use, as they typically require either the ability to compute $Z$ or the ability to easily draw samples from the model.
Kernelized Stein Discrepancy (KSD) emerged as a solution to this problem. As Lee et al. and Chwialkowski et al. independently demonstrated, KSD allows us to test how well a set of data samples fits a target distribution using only the score function—the gradient of the log-density. Because the gradient of $\log(p(x)/Z)$ is simply $\nabla \log p(x)$, the unknown constant $Z$ vanishes, enabling a nonparametric test of goodness-of-fit for complex, high-dimensional models.
The Intuition: Stein’s Identity
The core of this approach is Stein’s Identity. For a smooth probability density $p(x)$ and a sufficiently well-behaved function $f(x)$, the expectation of a specific operator $\mathcal{A}_p$ (the Stein operator) applied to $f$ is exactly zero if and only if the samples are drawn from $p$:
$$\mathbb{E}_{x \sim p} [\mathcal{A}_p f(x)] = 0$$
The Stein operator $\mathcal{A}_p f(x)$ is defined using the score function $\nabla \log p(x)$. Intuitively, if we have a set of samples $\{x_i\}$ and we find that the average value of $\mathcal{A}_p f(x_i)$ is far from zero, we have evidence that the samples did not come from $p$.
However, this raises a challenge: which function $f$ should we use to test the fit? A simple function might miss subtle differences between the data and the model. To make the test robust, we want to find the "worst-case" function $f$ that maximizes this deviation.
Kernelizing the Search
Searching over all possible functions $f$ is computationally impossible. This is where Reproducing Kernel Hilbert Spaces (RKHS) come in. By restricting $f$ to a unit ball within an RKHS, the authors of A Kernelized Stein Discrepancy for Goodness-of-fit Tests and A Kernel Test of Goodness of Fit showed that the optimal "witness function" can be found analytically.
The result is a closed-form expression for the discrepancy that can be computed as a simple U-statistic or V-statistic over the data samples. This "kernel trick" transforms a complex optimization problem into a tractable calculation involving a kernel function and the score of the target distribution. As Zhu et al. proved, these tests are "universally optimal," meaning they are guaranteed to detect any false hypothesis as the sample size grows.
From Testing to Optimization
While KSD was originally designed for goodness-of-fit testing, its ability to measure the distance to an unnormalized density makes it a powerful objective for optimization.
- Sampling and Variational Inference: Stein Variational Gradient Descent (SVGD) uses these principles to evolve a set of particles toward a target distribution. Salim et al. provided rigorous convergence theory for this process, showing how KSD acts as a functional gradient to drive particles toward high-density regions. Similarly, Korba et al. introduced KSD Descent, a deterministic method that minimizes KSD directly to generate high-quality samples.
- Training Implicit Models: In variational inference, we often want to use flexible "implicit" models where the density is unknown. Cheng et al. showed that KSD can serve as a training objective that avoids the biases of standard evidence lower bounds (ELBO).
- Generative Modeling: Researchers from the University of Tokyo applied KSD to train Generative Adversarial Networks (GANs) by measuring the distance to an intermediate target distribution, providing a more stable training signal than traditional adversarial losses.
Expanding the Domain
The versatility of the Stein-Kernel framework has led to its application across diverse data types and problems:
- Discrete and Structured Data: Qiang Liu extended the method to discrete distributions (Kernelized Discrete Stein Discrepancy), while Xu et al. and Weckbecker et al. adapted it for Exponential Random Graph Models, allowing for goodness-of-fit tests on complex network structures.
- Efficiency at Scale: To handle large datasets, Gorham et al. introduced Stochastic Stein Discrepancies, which use data subsampling to maintain rigorous convergence tracking while drastically reducing computational costs.
- Model Calibration: Glaser et al. utilized score-based kernels to verify if a model's uncertainty estimates are reliable, framing calibration as a conditional goodness-of-fit problem.
- Reinforcement Learning: In model-based RL, Bedi et al. used KSD to maintain a "coreset" of the most informative experiences, reducing memory bloat while maintaining performance.
By bridging the gap between theoretical probability (Stein's Method) and functional analysis (Kernels), this research direction provides a unified framework for evaluating and training models that were previously considered "black boxes" due to their intractable normalization.
Key Foundational Work
Black-box Importance Sampling (2017) — contributed 1 concept elements to this direction.
A Kernel Test of Goodness of Fit (2016) — contributed 1 concept elements to this direction.
A Kernelized Stein Discrepancy for Goodness-of-fit Tests (2016) — contributed 1 concept elements to this direction.