Data Analysis from Empirical Moments and the Christoffel Function

Edouard Pauwels, Mihai Putinar, Jean-Bernard LasserreFound. Comput. Math 2021 · OpenReview

Introduction

In modern data science, high-dimensional datasets are frequently assumed to reside on simpler, lower-dimensional structures such as manifolds or algebraic varieties (e.g., spheres, tori, or protein configuration spaces). However, standard statistical tools for density estimation often struggle when data is "singular," meaning it is restricted to a thin surface rather than being distributed across the entire ambient space. In such scenarios, the moment matrices used to describe the data become rank-deficient, which typically causes traditional algorithms to fail. This research bridges real algebraic geometry and machine learning to transform this mathematical "bug" into a feature, utilizing the spectral properties of these matrices for robust data analysis.

Methodology: Moments and the Christoffel Function

The authors propose a framework centered on two primary mathematical objects:

  1. The Empirical Moment Matrix ($M$): This matrix contains the averages of polynomial combinations of data points (e.g., $x, x^2, xy, \dots$).
  2. The Christoffel Function ($\Lambda$): Originating from approximation theory, this function acts as an inverse proxy for density. In regions where data is dense, the Christoffel function yields small values; in empty regions, the values are large.

While these tools are well-established for "nice," non-singular data, the authors extend them to the singular case. By employing the Moore-Penrose pseudo-inverse to handle rank-deficiency and applying results from algebraic geometry—specifically the Hilbert function—they demonstrate that the rank of the empirical matrix directly reveals the dimension of the data's support.

Geometric Insights

A core insight of this work is the "rigidity" of polynomials. Unlike many statistical methods that provide probabilistic approximations, the authors show that global geometric properties, such as dimension, can be inferred with probability one from a sufficiently large finite sample. The growth rate of the moment matrix's rank as the polynomial degree increases serves as a geometric "fingerprint": a 2D surface in 3D space shows quadratic rank growth, while a solid 3D object shows cubic growth.

Furthermore, the Christoffel function is particularly effective on manifolds without boundaries. On such shapes, a uniform data distribution results in a constant Christoffel function, making any deviations—indicating higher density areas—highly detectable.

Empirical Validation

The approach was validated across several synthetic and real-world datasets:

  • Synthetic Geometry: 20,000 points sampled from cubes, spheres, and tori to confirm dimension and rank detection.
  • Astronomy: Using 12,176 double star positions from the Hipparcos satellite, the method successfully identified the high-density band of the Milky Way.
  • Biochemistry: Analysis of 7,705 Glycine residues mapped "energetically allowed" regions of amino acid configurations on a bi-torus (a 4D object).

Conclusion and Future Directions

The paper establishes that the Christoffel function converges uniformly to the true density of the data, providing explicit convergence rates for singular sets like the sphere. However, practical implementation faces challenges. The method assumes data lies exactly on an algebraic set, raising questions about sensitivity to noise in real-world applications. Additionally, the size of the moment matrix grows combinatorially with the number of variables, suggesting that future work should focus on "kernelized" versions to improve scalability for high-dimensional problems.

Summary updated

BibTeX
@inproceedings{pauwels2021data,
  author = {Edouard Pauwels and Mihai Putinar and Jean-Bernard Lasserre},
  title = {{Data Analysis from Empirical Moments and the Christoffel Function}},
  booktitle = {Found. Comput. Math},
  year = {2021},
  url = {https://openreview.net/forum?id=zmnDbOHaMMb},
}
Figure