Maximum Likelihood Estimation in Gaussian Chain Graph Models under the Alternative Markov Property

Mathias Drton, Michael EichlerScandinavian 2006 · DOI · arXiv

Abstract

The Andersson–Madigan–Perlman (AMP) Markov property is a recently proposed alternative Markov property (AMP) for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced Lauritzen–Wermuth–Frydenberg Markov property that is coherent with data‐generation by natural block‐recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP chain graph models can be obtained by combining generalized least squares and iterative proportional fitting to an iterative algorithm. In an appendix, we give useful convergence results for iterative partial maximization algorithms that apply in particular to the described algorithm.

Introduction

Graphical models serve as a fundamental tool for representing conditional independence between variables. Chain graphs, in particular, offer a hybrid representation by incorporating both undirected edges (denoting symmetric relationships) and directed edges (denoting directional or causal relationships). Historically, the Lauritzen–Wermuth–Frydenberg (LWF) property was the standard interpretation for such graphs. However, the Andersson–Madigan–Perlman (AMP) property has emerged as a compelling alternative, especially when modeling data generated through block-recursive regressions—a process where groups of variables are sequentially determined by preceding groups.

While the statistical foundations for LWF models are well-established, a robust iterative algorithm for finding Maximum Likelihood Estimates (MLE) in Gaussian AMP models was previously lacking. This work addresses this gap by providing a formal iterative framework for parameter estimation and the calculation of standard errors.

Methodology

The joint distribution of a Gaussian AMP model can be factorized into a sequence of block-regressions. This factorization allows the global estimation problem to be decomposed into smaller, component-wise estimation tasks for each chain component of the graph.

The proposed approach utilizes an iterative partial maximization algorithm consisting of two primary steps:

  1. Regression Step: The regression coefficients ($\beta_\tau$) are estimated using a Generalized Least Squares (GLS) formulation, conditioned on a fixed covariance matrix.
  2. Covariance Step: The restricted covariance matrix ($\Omega_\tau$) is estimated via Iterative Proportional Fitting (IPF), conditioned on fixed regression coefficients.

The algorithm alternates between these steps until convergence. Theoretical analysis confirms that Gaussian AMP models belong to the class of curved exponential families, ensuring that the resulting MLEs are consistent and asymptotically normal.

Empirical Demonstration: University Graduation Rates

To validate the algorithm, we applied it to a dataset of $n = 159$ universities, originally used for college rankings. The model includes eight variables: student-teacher ratio (strat), average spending per student (spend), faculty salary (salar), rejection rate (rejr), test scores (tstsc), percent of students in the top 10% of their high school class (top10), percent accepted (pacc), and graduation percentage (apgra).

Figure 1: Chain graph for the university graduation rates dataset.

The graph structure (Figure 1) contains a non-decomposable chain component, making it a prime candidate for testing the iterative algorithm where simpler, two-step estimation methods (e.g., regression followed by residual covariance fitting) fail to yield the true MLE.

Results and Discussion

The application of the model to the university dataset resulted in a deviance of 16.89 with 11 degrees of freedom, suggesting a statistically sound fit. Notably, the iterative MLEs differed significantly from simpler two-step estimates; for instance, a specific regression coefficient shifted from 0.36 to 0.26 upon iteration. This highlights the necessity of the iterative approach for accurate parameter estimation in the presence of graph-imposed constraints.

While the algorithm efficiently finds solutions to the likelihood equations, it is important to note that it identifies local maxima; in highly complex models, the existence of multiple local maxima remains a possibility. Future research may extend these techniques to handle discrete or mixed variables under the AMP property, as well as more complex equality constraints across different blocks.

Summary updated

Abstract

The Andersson–Madigan–Perlman (AMP) Markov property is a recently proposed alternative Markov property (AMP) for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced Lauritzen–Wermuth–Frydenberg Markov property that is coherent with data‐generation by natural block‐recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP chain graph models can be obtained by combining generalized least squares and iterative proportional fitting to an iterative algorithm. In an appendix, we give useful convergence results for iterative partial maximization algorithms that apply in particular to the described algorithm.
BibTeX
@inproceedings{drton2006maximum,
  author = {Mathias Drton and Michael Eichler},
  title = {{Maximum Likelihood Estimation in Gaussian Chain Graph Models under the Alternative Markov Property}},
  booktitle = {Scandinavian},
  year = {2006},
  doi = {10.1111/j.1467-9469.2006.00482.x},
}
Figure