Measures with zeros in the inverse of their moment matrix

J. William Helton, Jean B. Lasserre, Mihai PutinarThe Annals of Probability 2008 · DOI

TL;DR

This paper explores the conditions under which zeros appear in the inverse of truncated moment matrices for multivariate measures. The authors establish a precise link between these zeros and the "product form property" of the measure, which is equivalent to independence of random variables. This connection is further elucidated through the study of orthonormal polynomials, revealing that zeros in the inverse matrix correspond to a specific "conditional triangularity" property of these polynomials.

What Problem Does This Paper Address?

In statistics and probability, understanding the structure of multivariate distributions is crucial. The covariance matrix, a fundamental tool, can be viewed as a matrix of second moments. When dealing with complex models, especially in Bayesian estimation and graphical models, it's beneficial to exploit sparsity in the inverse covariance matrix. This paper generalizes this idea to truncated moment matrices of arbitrary measures. The core problem is to precisely characterize when specific entries in the inverse of these moment matrices are zero, and to understand the probabilistic or structural meaning of these zeros. Specifically, the authors aim to identify when these zeros are a consequence of the measure having a "product structure" (i.e., independence of its constituent random variables).

Methods

The paper employs a functional analytic approach, working with linear functionals on polynomial spaces rather than solely with measures. Key techniques include:

  • Moment Matrices: Construction and analysis of truncated moment matrices $M_d(y)$ from a sequence of moments $y$.
  • Orthonormal Polynomials: Development and study of orthonormal polynomials associated with a positive linear functional. The paper leverages the Gram-Schmidt process and the graded lexicographic order (GLex) to define and work with these polynomials.
  • Triangularity Properties: The central concept is "conditional triangularity" of orthonormal polynomials. This property, when applied to polynomials in a subset of variables while treating others as parameters, is shown to be equivalent to zeros in the inverse moment matrix.
  • Matrix Completion: The problem is related to constrained matrix completion, where one seeks to fill in missing entries of a moment matrix while maintaining positive definiteness and respecting the moment structure.
  • Induction and Linear Algebra: Proofs often rely on induction and direct algebraic manipulation of matrix entries and polynomial coefficients.

Datasets

Not discussed in the paper. The paper is theoretical and focuses on the properties of measures and their associated moment matrices and orthogonal polynomials, rather than empirical evaluation on specific datasets.

Key Insights and Ideas

  • Zeros in Inverse = Independence: The paper establishes a direct link between zeros in specific entries of the inverse truncated moment matrix and the independence of the underlying random variables (manifested as a "product form property" of the measure).
  • Conditional Triangularity is Key: The crucial insight is that the presence of zeros in the inverse moment matrix is precisely characterized by a "conditional triangularity" property of the associated orthonormal polynomials. This means that when a subset of variables is fixed, the polynomials in the remaining variables exhibit a specific triangular structure.
  • Generalization Beyond Gaussian: While zeros in the inverse covariance matrix are well-understood for Gaussian distributions (related to partial correlation), this work extends these ideas to non-Gaussian measures and higher-order moments.
  • Product Structure vs. General Independence: The paper distinguishes between a "product form property" (leading to independence) and a more general "partial independence property" for the measure, showing how these translate to specific zero patterns in the inverse moment matrix.
  • Orthogonal Polynomials as a Bridge: Orthonormal polynomials serve as a powerful intermediary, translating structural properties of the measure (like independence) into algebraic properties (like triangularity) that can be analyzed in the context of moment matrices.

Key Results

  • Theorem 3.1: A pair of multi-indices $(\alpha, \beta)$ corresponds to a "congenital zero" (a zero arising from independence) in $M_d(y)^{-1}$ if and only if the least common multiple of $X^\alpha$ and $X^\beta$ has a degree greater than $d$. This is rephrased using the $\max(\alpha, \beta)$ operation.
  • Theorem 4.1: Generalizes Theorem 3.1 to "partial independence" for grouped variables, linking zeros in $M_d(y)^{-1}$ to a specific structure of orthonormal polynomials across these groups.
  • Theorem 5.1 (Full Triangularity): Establishes that zeros in $M_r(y)^{-1}$ for $r < d$ (whenever $|\max(\alpha, \beta)| > r$) are equivalent to the associated orthonormal polynomials having "full triangular form."
  • Theorem 6.1 (Conditional Triangularity): This is a central result, showing that the "zero in the inverse condition" (related to zeros in $M_r(y)^{-1}$) is equivalent to the orthonormal polynomials satisfying "conditional triangularity" with respect to a subset of variables.
  • Corollary 6.2: For $d=1$, this shows that zero partial correlation between two variables $X_i$ and $X_j$ (given others) is equivalent to the orthonormal polynomials up to degree 2 satisfying conditional triangularity with respect to $X = (X_k)_{k \neq i,j}$.

Limitations and Future Work

  • Non-Gaussian Case Complexity: The paper notes that for non-Gaussian distributions, conditional independence does not generally imply zeros in the inverse of the covariance matrix (or moment matrix). This suggests that while the paper provides necessary and sufficient conditions for zeros related to independence, the converse (zeros implying conditional independence) is not universally true outside specific cases (like Gaussian).
  • Computational Aspects: While the paper provides theoretical characterizations, the direct computation of these orthonormal polynomials and moment matrices can be complex, especially for high degrees and dimensions.
  • Future Directions: The paper raises questions about when zero partial correlation implies conditional independence in the non-Gaussian case. It also suggests exploring the relationship between conditional independence and the zero inverse property for $d > 1$. The authors also hint at further connections to matrix completion problems and the interpretation of "physical" or probabilistic meaning of solutions in that context.

Summary updated

Abstract

We investigate and discuss when the inverse of a multivariate truncated moment matrix of a measure μ has zeros in some prescribed entries. We describe precisely which pattern of these zeroes corresponds to independence, namely, the measure having a product structure. A more refined finding is that the key factor forcing a zero entry in this inverse matrix is a certain conditional triangularity property of the orthogonal polynomials associated with μ.
BibTeX
@article{helton2008measures,
  author = {J. William Helton and Jean B. Lasserre and Mihai Putinar},
  title = {{Measures with zeros in the inverse of their moment matrix}},
  journal = {The Annals of Probability},
  year = {2008},
  doi = {10.1214/07-aop365},
}
Figure