Distribution Testing in High Dimensions

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2115 papers158 conceptsfrozen 2026-10-04

In modern machine learning, we frequently need to answer a deceptively simple question: are these two sets of data the same? This "two-sample problem" underpins everything from detecting data drift in production models to evaluating whether a GAN has successfully captured a target distribution. A closely related challenge is the "goodness-of-fit" (GoF) test, where we ask if a set of samples matches a specific mathematical model.

While these problems are well-understood for simple, low-dimensional data, they become notoriously difficult as the number of features grows. This research direction focuses on redesigning statistical tests to survive the "curse of dimensionality."

The Vanishing Signal in High Dimensions

The fundamental challenge in high-dimensional testing is that our standard tools for measuring "distance" between distributions lose their edge. In a low-dimensional space, it is easy to see if two clusters of points overlap. However, as Ery Arias-Castro and Bruno Pelletier argue in Remember the Curse of Dimensionality, the minimum detectable difference between distributions degrades exponentially as dimensions increase.

Traditional metrics like the Maximum Mean Discrepancy (MMD) or the Kernelized Stein Discrepancy (KSD) introduced by Jason D. Lee and Qiang Liu rely on kernels to compare points. In high dimensions, these kernels often decay too quickly, causing the "signal" of the difference to be swallowed by the "noise" of the ambient space. As Andreas Anastasiou and colleagues note, this leads to a massive loss of statistical power—the test simply fails to reject the null hypothesis even when the distributions are clearly different.

Learning Where to Look

If generic kernels fail because they look at every dimension equally, the logical solution is to learn which dimensions or features actually matter.

Feng Liu and Wenkai Xu pioneered the use of Deep Kernels for Non-Parametric Two-Sample Tests. Instead of using a fixed Gaussian kernel, they use a neural network to extract features from the data before performing the MMD test. By optimizing the network to maximize the test's power, the model learns to ignore irrelevant high-dimensional noise and focus on the specific "semantic" differences between datasets.

A similar intuition drives Classifier Two-Sample Tests (C2ST). David Lopez-Paz and Maxime Oquab reframed distribution testing as a classification problem: if a powerful neural network cannot distinguish between two datasets better than a random coin flip, the datasets are effectively identical. This bypasses the need for manual kernel selection entirely.

Slicing and Projecting

Another way to break the curse of dimensionality is to reduce the problem back to one dimension, where statistical tests are most powerful.

The Projected Wasserstein Distance, developed by Jie Wang and Rui Gao, works by finding the single best linear projection that maximizes the distance between two distributions. By focusing on this "worst-case" projection, the test maintains sensitivity even when the distributions only differ in a tiny subspace of the data.

Similarly, Kaibo Zhang and Yingzhen Li explored Active Slices for Sliced Stein Discrepancy. Rather than averaging over all possible directions (which introduces noise), they search for "active" directions where the discrepancy is most apparent.

Exploiting Data Geometry

Real-world high-dimensional data, such as images or text embeddings, rarely fills the entire space. Instead, it usually sits on a low-dimensional "manifold."

Xiuyuan Cheng demonstrated that Kernel MMD tests can overcome the curse of dimensionality if the kernel is properly scaled to the intrinsic dimension of this manifold. By respecting the underlying geometry, we can detect subtle differences that would be invisible if we treated the data as purely high-dimensional. Jiaqi Gu and Guosheng Yin took this further with the Delaunay Weighted Two-sample Test, which uses the triangulation of data points to capture relative directions and local density, providing a significant power boost for complex datasets like human faces.

Testing Without Normalization

A recurring theme in this field is the use of Stein's Method. In many Bayesian and generative modeling contexts, we know the "shape" of a distribution but not its "scale" (the normalizing constant). This makes traditional GoF tests impossible.

The Kernelized Stein Discrepancy (KSD) solves this by using a "Stein operator" that only requires the gradient of the log-density. This allows researchers like Will Grathwohl to evaluate energy-based models without needing to sample from them, a major computational hurdle in deep generative modeling. However, as Xing Liu points out, even KSD can be "blind" to differences when distributions have multiple distant modes, leading to ongoing work in perturbed and polynomial Stein discrepancies to ensure these tests remain robust in the most challenging high-dimensional landscapes.

Papers in this version

Showing 60 of 115 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 108 papers
  2. Version 1 · 2026-10-04 · 115 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_high_dimensional_power_decay_3e609ead,
  title = {high-dimensional power decay},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-dimensional-power-decay/rdr_3e609ead62c549ae8247f7806c01559f},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_distribution_testing_in_high_dimensions_3e609ead_v1,
  title = {Distribution Testing in High Dimensions},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/distribution-testing-in-high-dimensions/rdr_3e609ead62c549ae8247f7806c01559f/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}