When fitting a statistical model using Maximum Likelihood Estimation (MLE), we generally assume that our optimization algorithms will converge to a valid set of parameters. However, for highly structured models—such as Gaussian graphical models or hierarchical log-linear models—a valid MLE is not guaranteed to exist. If the data is too sparse or sample sizes are too small, the likelihood function may be unbounded, or its maximum may lie on the boundary of the parameter space where the model breaks down (e.g., where variances go to zero or probabilities become exactly zero).
The core insight unifying this research direction is that MLE existence is fundamentally a geometric problem. Instead of viewing estimation as an analytic optimization task, we can determine whether an MLE exists by looking at where the data's sufficient statistics reside relative to a specific convex geometric space. If the observed statistics lie strictly within the relative interior of this space, a unique MLE exists; if they fall on the geometric boundary, the estimation fails.
Convex Cones and Matrix Completion in Gaussian Models
For multivariate Gaussian models, the parameters we want to estimate are typically constrained within the space of positive definite matrices. The sufficient statistics extracted from our data are summarized in a sample covariance matrix.
Sturmfels et al. framed this relationship through the lens of convex algebraic geometry. They demonstrated that the cone of concentration (inverse covariance) matrices and the cone of sufficient statistics are convex duals. This duality reveals that finding an MLE is mathematically identical to a positive definite matrix completion problem: given a sample covariance matrix with missing or unconstrained entries, can we fill in the blanks such that the entire matrix becomes positive definite?
[ Valid Parameter Cone ] <--- Dual Cones ---> [ Sufficient Statistic Cone ]
(Concentration Matrices) (Sample Covariances)
|
Is the data in the interior?
Yes -> MLE Exists
No -> Boundary / Failure
Building on this geometric duality, Uhler analyzed the exact boundaries of these spaces for Gaussian graphical models. The MLE exists if and only if the sample statistics fall strictly within the interior of the convex cone of valid covariance matrices. If the number of data observations is too low, the sample statistics are forced onto the boundary of the cone, meaning no valid positive definite completion exists. By utilizing polynomial ideals to define these boundaries algebraically, Uhler showed that the MLE can sometimes exist even when the number of samples is smaller than traditionally expected by graph-theoretic bounds, shifting the problem from combinatorics to pure geometry.
Polyhedral Geometry and Sparse Discrete Data
When we transition from continuous Gaussian models to discrete log-linear models, the smooth, curved boundaries of algebraic cones become flat-faced polyhedral shapes known as marginal cones.
In hierarchical log-linear models, the sufficient statistics are the observed marginal counts of a contingency table. Eriksson et al. demonstrated that the MLE exists if and only if these observed data margins fall within the relative interior of the model's marginal cone.
Intuition for this can be drawn from data sparsity. When a dataset contains many zero counts, the vector of observed margins is pushed away from the center of the polyhedral cone and lands directly on one of its boundary faces. By framing this boundary condition via polyhedral geometry and Minkowski sums, the authors showed that checking for MLE existence does not require running an optimization algorithm to failure. Instead, it can be computed beforehand using linear programming to verify if the data vector sits safely away from the polyhedral walls.
The Algebraic Bridge: Invariant Theory and Stability
While convex cones and polyhedra offer excellent visual intuition for specific model classes, a broader question remains: is there a unified framework that explains MLE existence across both discrete and continuous models?
The answer is found in Geometric Invariant Theory (GIT), a branch of algebraic geometry that studies the behavior of spaces under group actions. Améndola et al. established a direct dictionary between log-linear maximum likelihood estimation and toric invariant theory. They proved that finding the MLE is mathematically equivalent to finding a point of minimal norm within an orbit under a torus action.
Under this framework, the geometric position of the data translates directly into algebraic notions of stability:
| Stability Level in Invariant Theory | Statistical Property of the MLE |
|---|---|
| Unstable | The likelihood function is unbounded. |
| Semistable | The likelihood is bounded, but no MLE achieves it within the space. |
| Polystable | A unique MLE is guaranteed to exist. |
| Stable | A unique MLE exists and possesses a well-behaved, closed orbit. |
In a subsequent work, Améndola et al. extended this invariant theory framework to Gaussian models and matrix scaling algorithms. They showed that classical iterative scaling algorithms used by statisticians are actually optimization routines designed to minimize these geometric norms.
By unifying these perspectives, this research direction demonstrates that whether you are completing a covariance matrix, counting contingency tables, or scaling tensors, statistical estimation succeeds only when the underlying geometry of the data keeps it clear of the model's boundaries.
Key Foundational Work
Polyhedral conditions for the nonexistence of the MLE for hierarchical log-linear models (2006) — contributed 6 concept elements to this direction.
Multivariate Gaussians, Semidefinite Matrix Completion, and Convex Algebraic Geometry (2009) — contributed 4 concept elements to this direction.