Abstract: We analyze the problem of maximum likelihood estimation for Gaussian distributions that are multivariate totally positive of order two ($\mathrm{MTP}_{2}$). By exploiting connections to phylogenetics and single-linkage clustering, we give a simple proof that the maximum likelihood estimator (MLE) for such distributions exists based on $n\geq2$ observations, irrespective of the underlying dimension. Slawski and Hein [Linear Algebra Appl. 473 (2015) 145–179], who first proved this result, also provided empirical evidence showing that the $\mathrm{MTP}_{2}$ constraint serves as an implicit regularizer and leads to sparsity in the estimated inverse covariance matrix, determining what we name the ML graph. We show that we can find an upper bound for the ML graph by adding edges corresponding to correlations in excess of those explained by the maximum weight spanning forest of the correlation matrix. Moreover, we provide globally convergent coordinate descent algorithms for calculating the MLE under the $\mathrm{MTP}_{2}$ constraint which are structurally similar to iterative proportional scaling. We conclude the paper with a discussion of signed $\mathrm{MTP}_{2}$ distributions.
Introduction
In modern high-dimensional statistics, researchers frequently encounter the "$p > n$" problem, where the number of variables exceeds the number of observations. In such scenarios, the standard Gaussian maximum likelihood estimator (MLE) for the covariance matrix does not exist. While $L_1$-regularization methods like the Graphical Lasso are widely used to address this, they require careful tuning of penalty parameters. This paper explores a different approach: imposing the structural constraint of Multivariate Total Positivity of order two ($MTP_2$).
A Gaussian distribution is $MTP_2$ if its concentration (precision) matrix is an M-matrix, characterized by non-positive off-diagonal elements. This implies that all partial correlations between variables are non-negative. While seemingly restrictive, $MTP_2$ is a natural property in various fields, including biology and psychology. The authors demonstrate that this constraint acts as an implicit regularizer, ensuring MLE existence even with minimal data and producing sparse, interpretable models.
Theoretical Breakthrough: Existence for $n \ge 2$
A key theoretical contribution of this work is the proof that the $MTP_2$ MLE exists and is unique as long as $n \ge 2$, regardless of the number of variables $p$. This is a significant improvement over the unconstrained case, which requires $n \ge p$. The authors achieve this by linking the problem to the theory of ultrametrics and single-linkage clustering, providing a simpler and more elegant proof than previous literature.
The ML Graph and Implicit Regularization
The $MTP_2$ constraint naturally induces sparsity in the precision matrix. The resulting sparsity pattern is termed the ML graph. Unlike the Lasso, which uses an artificial penalty, $MTP_2$ creates sparsity as a mathematical consequence of the M-matrix constraint.
The authors show that the ML graph is bounded by the Maximum Weight Spanning Forest (MWSF) of the sample correlation matrix plus "excess correlations" that the forest cannot explain. This provides a clear geometric intuition: the model starts with a tree-like backbone and only adds edges that represent significantly stronger correlations than those accounted for by the forest.
Computational Framework
The paper proposes two globally convergent coordinate descent algorithms to compute the $MTP_2$ MLE. These algorithms iteratively update pairs of variables using closed-form solutions for $2 \times 2$ sub-problems. This approach is computationally efficient and shares structural similarities with iterative proportional scaling, making it suitable for large-scale applications.
Applications and Signed $MTP_2$
The framework is validated using two primary datasets:
- Carcass Data: Measurements of pig carcasses are used to illustrate how the $MTP_2$ constraint recovers a sparse physical structure (see Figure 1). The resulting graph naturally prunes weak or "frustrated" edges.
- Personality Traits: In psychometrics, some traits are naturally negatively correlated. The authors introduce a Signed $MTP_2$ heuristic, which identifies sign-flips to transform the data into a positive dependence structure. This approach improves model interpretability and connects the statistical framework to "non-frustrated" systems in physics.
Conclusion
By establishing the existence of the MLE for $n \ge 2$ and characterizing the resulting ML graph, this paper provides a robust foundation for Gaussian modeling under total positivity. The $MTP_2$ constraint offers a parameter-free path to sparsity and interpretability, particularly valuable in high-dimensional settings where positive dependence is expected.
Summary updated