Interpretable Stein Goodness-of-fit Tests on Riemannian Manifold

Wenkai Xu, Takeru MatsudaICML 2021

Introduction

This paper introduces Manifold Kernel Stein Discrepancy (mKSD), a novel framework for testing how well a statistical model fits data residing on non-Euclidean spaces like spheres, tori, or rotation groups. By extending Stein’s method to Riemannian manifolds, the authors develop "normalization-free" tests that can evaluate complex, unnormalized probability densities while providing interpretable feedback on exactly where a model fails to capture the data's structure.

The Challenge of Non-Euclidean Spaces

In many fields—from robotics (rotation groups) to biology (protein angles on a torus)—data doesn't live in flat Euclidean space. Standard statistical tests fail here because they don't account for the "curves" and boundaries of these manifolds.

Furthermore, many powerful probabilistic models are defined by unnormalized densities. They tell you the relative likelihood of points, but calculating the total "volume" (the normalization constant) is often computationally impossible. Existing goodness-of-fit tests for manifolds either require this constant or are restricted to very specific distributions (like the uniform distribution). There hasn't been a general-purpose, computationally efficient way to perform model criticism on Riemannian manifolds—until now.

Methodology: Manifold Kernel Stein Discrepancy

The authors bridge the gap between Stein’s Method (a tool for characterizing distributions via operators) and Riemannian Geometry.

  1. Manifold Stein Operators: Using Stokes’ Theorem, they derive three flavors of Stein operators:
  2. First-order ($A^{(1)}$): Uses gradients and is generally the most efficient.
  3. Second-order ($A^{(2)}$): Involves the metric tensor (the "ruler" of the manifold); useful for specific geometric contexts but more expensive to compute.
  4. Zeroth-order ($A^{(0)}$): Equivalent to Maximum Mean Discrepancy (MMD), but requires the ability to sample from the model.
  5. mKSD Statistic: They combine these operators with Reproducing Kernel Hilbert Space (RKHS) theory to create a test statistic that is zero if and only if the model perfectly matches the data distribution. Crucially, for the 1st and 2nd order versions, the normalization constant drops out of the math.
  6. Interpretable Model Criticism (mFSSD): They propose a "Finite Set" version of the test. By optimizing a small set of "test locations" on the manifold to maximize the test power, they can visualize exactly where the model deviates from the observed data.
  7. Wild Bootstrap: To determine if a result is statistically significant (p-value), they use a wild bootstrap procedure to approximate the null distribution of the test statistic.

Datasets Evaluated

The paper evaluates the proposed tests on both synthetic and real-world datasets:

  • Synthetic SO(3): Simulations on the rotation group using Uniform and Fisher distributions to test the power of the method against varying levels of perturbation.
  • Vectorcardiogram Data: A real-world dataset recording 3D perpendicular vectors of heart electrical activity (QRS, PRS, and T waves) from children.
  • Tokyo Wind Direction: Bivariate circular data (representing wind at 00:00 and 12:00) mapped onto a 2D torus.

Key Insights

  • Diffusion Along the Surface: A key conceptual "aha" is that these manifold Stein operators "diffuse" information strictly along the surface of the manifold. A standard Euclidean operator would try to move "off" the sphere or "through" the torus, leading to incorrect results.
  • The Power of First-Order: The authors prove (via Bahadur efficiency) that the first-order operator is generally more powerful than the second-order one. It captures richer distributional features while being cheaper to compute ($O(n^2d)$ vs $O(n^2d^3)$).
  • Interpretability via Optimization: By treating the test locations as parameters to be optimized, the test becomes a diagnostic tool. If a model for wind patterns fails, the mFSSD can point to specific angle combinations where the model's predictions are most "wrong."

Key Results

  • Superior Power: In SO(3) simulations, mKSD1 consistently outperformed the classical Sobolev test and MMD-based tests, especially as the model became more complex or the sample size increased.
Test Power Comparison
  • Heart Data Insights: The tests successfully rejected a standard Fisher distribution for the vectorcardiogram data (p-values < 0.01), whereas the traditional Sobolev test failed to find the misfit (p=0.126).
  • Capturing Interactions: On the Tokyo wind data, the mKSD test correctly identified that a "no-interaction" model was insufficient, proving there is a significant statistical dependency between midnight and noon wind directions.

Limitations and Future Work

  • Metric Tensor Complexity: The second-order operator requires computing the inverse of the metric tensor, which can be computationally heavy for high-dimensional or mathematically complex manifolds.
  • Kernel Selection: While the authors provide a way to optimize kernel parameters, the choice of the base kernel (e.g., the "exponential-trace" kernel) still requires some domain expertise regarding the manifold's topology.
  • Future Directions: The authors suggest exploring how different mKSDs induce different "topologies" on the space of distributions, which could lead to new ways of training generative models on manifolds.

Summary updated

BibTeX
@inproceedings{xu2021interpretable,
  author = {Wenkai Xu and Takeru Matsuda},
  title = {{Interpretable Stein Goodness-of-fit Tests on Riemannian Manifold}},
  booktitle = {International Conference on Machine Learning},
  year = {2021},
}
Figure