The maximum likelihood threshold of a path diagram

Mathias Drton, Christopher Fox, Andreas Käufl, Guillaume PouliotThe Annals of Statistics 2019 · DOI

TL;DR

This paper introduces the "maximum likelihood threshold" for Gaussian structural equation models, defining the minimum sample size required for the model's likelihood function to be well-behaved. They show this threshold is surprisingly low even for models with feedback loops, making likelihood-based inference more broadly applicable to complex, high-dimensional systems.

What Problem Does This Paper Address?

Linear structural equation models (SEMs) are powerful tools for understanding relationships between variables, often visualized using "path diagrams." These diagrams can include directed edges (representing causal influences) and bidirected edges (representing unmeasured common causes or correlations between error terms). A key challenge in using SEMs, especially in high-dimensional settings where the number of variables is large, is ensuring that the statistical inference methods, particularly maximum likelihood estimation, are well-defined. Specifically, the likelihood function might not have a finite maximum, meaning it can grow infinitely large, rendering standard estimation impossible. This paper addresses the question: what is the minimum sample size needed to guarantee that the likelihood function does have a finite maximum (is bounded above)?

Methods

The core of the paper is the derivation of the "maximum likelihood threshold" (mlto) for a given path diagram. They leverage the graphical structure of the path diagram, decomposing it based on its bidirected components.

  • Graphical Representation: Path diagrams are used to encode the structure of the SEM. Directed edges represent linear dependencies, and bidirected edges represent correlations among error terms.
  • Decomposition: The key insight is to decompose the path diagram based on the connected components of its bidirected part. This allows them to analyze the problem by considering simpler subgraphs.
  • Threshold Calculation: For acyclic digraphs (models without feedback loops), the threshold is related to the maximum in-degree of any node. For general mixed graphs (allowing feedback loops and bidirected edges), the threshold is determined by the size of the "parent sets" of the connected components of the bidirected graph.
  • Proof Techniques: The proofs involve analyzing the properties of the covariance matrix, using concepts like positive definiteness, and employing matrix identities (e.g., Woodbury matrix identity) and inequalities (e.g., Hadamard inequality).

Datasets

Not discussed in the paper. The paper focuses on theoretical results and uses simulated examples for illustration.

Key Insights and Ideas

  • Feedback Loops Don't Necessarily Inflate Thresholds: Surprisingly, the presence of feedback loops (directed cycles) in the path diagram does not inherently lead to a high maximum likelihood threshold. The threshold for acyclic digraphs is related to the maximum in-degree.
  • Bidirected Edges are the Key Drivers of High Thresholds: Bidirected edges, representing unmeasured confounders or correlated errors, are the primary drivers of potentially large maximum likelihood thresholds. These can create "long paths" in the bidirected structure.
  • Decomposition is Powerful: Breaking down the complex graph into components related to its bidirected structure simplifies the problem and reveals the underlying structure determining the threshold.
  • Guaranteed Boundedness: The paper provides a concrete threshold: if the sample size is below this threshold, the likelihood is almost surely unbounded. If it's at or above, it's almost surely bounded. This is crucial for guaranteeing that likelihood-based estimation will work.

Key Results

  • Theorem 2: For any mixed graph G, the maximum likelihood threshold (mlto(G)) is determined by the maximum size of the "parent set" of the connected components of its bidirected part. Specifically, if $C_1, \dots, C_l$ are the vertex sets of the connected components of the bidirected part of G, then $mlto(G) = \max_{j=1,\dots,l} |Pa(C_j)|$.
  • Unboundedness Below Threshold: If the sample size $n < mlto(G)$, then the likelihood function is almost surely unbounded.
  • Special Case (Acyclic Digraphs): For acyclic digraphs (no feedback loops), the threshold simplifies to $1 + \max_j |pa(j)|$, where $pa(j)$ is the set of parents of node $j$.

Limitations and Future Work

  • Achieving the Maximum: While the paper guarantees the likelihood function is bounded above for $n \ge mlto(G)$, it doesn't guarantee that the maximum is achieved (i.e., that a maximizer exists). This is particularly relevant for non-closed parameter spaces, which can occur with mixed graphs. The authors note this is an important area for future research.
  • General Mixed Graphs: The paper provides a clear threshold for general mixed graphs, but the "achievability" of the maximum remains an open question for some cases.
  • Numerical Experiments: The paper includes a numerical experiment demonstrating that even for high-dimensional, sparse, cyclic models, likelihood inference can be effective at sample sizes below what might be naively expected, aligning with their theoretical findings. This suggests practical applicability.

Summary updated

Abstract

Linear structural equation models postulate noisy linear relationships between variables of interest. Each model corresponds to a path diagram, which is a mixed graph with directed edges that encode the domains of the linear functions and bidirected edges that indicate possible correlations among noise terms. Using this graphical representation, we determine the maximum likelihood threshold, that is, the minimum sample size at which the likelihood function of a Gaussian structural equation model is almost surely bounded. Our result allows the model to have feedback loops and is based on decomposing the path diagram with respect to the connected components of its bidirected part. We also prove that if the sample size is below the threshold, then the likelihood function is almost surely unbounded. Our work clarifies, in particular, that standard likelihood inference is applicable to sparse high-dimensional models even if they feature feedback loops.
BibTeX
@article{drton2019maximum,
  author = {Mathias Drton and Christopher Fox and Andreas Käufl and Guillaume Pouliot},
  title = {{The maximum likelihood threshold of a path diagram}},
  journal = {The Annals of Statistics},
  year = {2019},
  doi = {10.1214/18-aos1724},
}

Loading related content...

Figure