In modern Bayesian inference and generative modeling, we often encounter "unnormalized" probability distributions. These are models where we know the relative shape of the density—the numerator—but calculating the total volume (the "partition function" or normalization constant) is computationally impossible. This makes standard statistical tools like the Kullback-Leibler divergence or Maximum Mean Discrepancy (MMD) difficult to use, as they typically require either the full density or the ability to easily sample from the model.
Score-based discrepancies, pioneered by researchers like Qiang Liu, Lester Mackey, and Andrew Duncan, provide a way to bypass this "normalization trap." By looking at the gradient of the log-density—the score function—rather than the density itself, these methods allow us to compare data to models without ever needing to know the normalization constant.
The Score Function as a Normalization-Free Compass
The core intuition behind this research direction is that the normalization constant is a global property, but the "shape" of a distribution is local. If you take the derivative of the log-density $\log p(x) = \log f(x) - \log Z$, the constant $Z$ vanishes: $$\nabla_x \log p(x) = \nabla_x \log f(x)$$ This gradient, the score function, acts like a compass pointing toward the high-probability regions of the distribution. As Jason D. Lee, Qiang Liu, and Michael Jordan demonstrated in A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation, we can use this score function to build a "goodness-of-fit" test. If a set of data points truly comes from a model, the points should, on average, align with the model's score function in a very specific way.
Stein’s Identity: The Zero-Mean Property
The mathematical engine of this approach is Stein’s Identity. For a target distribution $p$, we can define a "Stein operator" $\mathcal{A}_p$. This operator has a remarkable property: for a wide class of functions $f$, the expectation of $\mathcal{A}_p f(x)$ is exactly zero if and only if the data $x$ is drawn from $p$.
$$\mathbb{E}_{x \sim p} [\mathcal{A}_p f(x)] = 0$$
If the data comes from a different distribution $q$, this expectation will generally not be zero. The "Stein Discrepancy" is defined as the maximum possible violation of this identity. As Jackson Gorham and Lester Mackey showed in Measuring Sample Quality with Stein's Method, this provides a rigorous way to measure how well a set of samples represents a target distribution, even when those samples are biased or generated by complex MCMC chains.
Kernelization: Turning Theory into Computation
To turn Stein's identity into a practical algorithm, we need a way to search for the function $f$ that most strongly highlights the mismatch between the data and the model. By choosing $f$ from a Reproducing Kernel Hilbert Space (RKHS), we can derive a closed-form solution for the discrepancy.
This Kernelized Stein Discrepancy (KSD) allows practitioners to calculate a single number that represents the distance between a dataset and an unnormalized model. This has several advantages:
- No Sampling Required: Unlike MMD, which requires samples from both the data and the model, KSD only requires samples from the data and the score function of the model. This avoids the need for expensive MCMC sampling during testing (Jason D. Lee et al., 2016).
- Analytic Gradients: Because KSD is a differentiable function of the sample positions, we can use it as a loss function. This led to the development of Stein Variational Gradient Descent (SVGD) and KSD Descent (Anna Korba et al., 2021), where particles are iteratively moved to minimize the discrepancy and "fit" the target distribution.
Scaling and Specialization
While the original KSD scales quadratically with the number of samples ($O(n^2)$), recent work has focused on making these tests faster and more versatile. Wittawat Jitkrittum introduced the Finite Set Stein Discrepancy (FSSD) in A Linear-Time Kernel Goodness-of-Fit Test, which achieves linear-time complexity by evaluating the score function at a small number of optimized "test locations."
The framework has also been extended beyond simple Euclidean spaces:
- Manifolds: Wenkai Xu and Takeru Matsuda developed Manifold KSD to test models on spheres and tori (Interpretable Stein Goodness-of-fit Tests on Riemannian Manifold).
- Sequential Data: Jerome Baum extended the method to variable-length sequences like text or DNA (A Kernel Stein Test of Goodness of Fit for Sequential Models).
- Discrete Graphs: Moritz Weckbecker and colleagues explored how different kernels affect the testing of random graph generators (On RKHS Choices for Assessing Graph Generators via Kernel Stein Statistics).
Applications in Modern ML
Today, Stein discrepancies are used as a "Swiss Army knife" for unnormalized models. In Reinforcement Learning, Amrit Bedi and Souradip Chakraborty use KSD to maintain "coresets"—compressed versions of experience buffers that retain only the most informative transitions (Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning). In Bayesian Computation, Marina Riabiz and Lester Mackey use it for "Stein Thinning," a method to pick the best subset of MCMC samples to reduce storage costs without losing accuracy (Optimal Thinning of MCMC Output).
By shifting the focus from "how likely is this point?" to "how well does this point's local gradient match the model's score?", Stein-based methods have provided a robust, theoretically grounded, and computationally efficient path for evaluating the complex, unnormalized models that define modern AI.