Binary Models for Marginal Independence

Mathias Drton, Thomas S. RichardsonJournal of the Royal Statistical Society Series B (Statistical Methodology) 2008 · DOI

TL;DR

This paper introduces a new class of graphical models, called binary bi-directed graph models, specifically designed to represent marginal independence relationships in binary data. These models overcome limitations of traditional log-linear models, offering a flexible framework for analyzing complex dependencies and enabling efficient maximum likelihood estimation.

What Problem Does This Paper Address?

Analyzing contingency tables, especially those with binary variables, often involves understanding conditional independences. Traditional graphical log-linear models are excellent for this, but they struggle to directly represent marginal independences, which occur when variables are independent after accounting for other factors (e.g., confounding). This limitation arises because log-linear models parameterize the full joint distribution, making it difficult to isolate specific marginal relationships. The authors aim to bridge this gap by developing a graphical modeling framework analogous to multivariate Gaussian models for marginal independence, but tailored for binary data.

Methods

The core of the paper lies in introducing binary bi-directed graph models. These models use bi-directed graphs (where edges can be traversed in both directions) to represent marginal independence structures. The authors leverage an analogy with multivariate Gaussian models, translating their graphical representations to the binary setting.

Key methodological contributions include:

  • Graphical Representation: Using bi-directed graphs to encode marginal independence hypotheses.
  • Möbius Parameterization: A novel parameterization using Möbius parameters (derived from joint cell probabilities) that simplifies the representation of these independence constraints. This parameterization is shown to be bijective and allows for efficient computation.
  • Iterative Conditional Fitting (ICF): An adaptation of an existing algorithm for Gaussian models, ICF is extended to the binary case for maximum likelihood estimation. This algorithm iteratively updates parameters for each variable while keeping others fixed, ensuring the model constraints are maintained.
  • Incorporating Symmetry: The framework is extended to handle symmetry restrictions, which are common in data from paired individuals (like twins) or repeated measurements.

Datasets

The paper uses two primary datasets for illustration and validation:

  • Twin Data: Data from a study on monozygotic twins, examining alcohol dependence and depression status. This dataset is used to illustrate the core concepts and the challenge of representing marginal independence.
  • U.S. General Social Survey (GSS) Data: Data from a survey on trust in social institutions and membership in organizations. This dataset is used to demonstrate the application of the binary bi-directed graph models in a real-world scenario.

Key Insights and Ideas

  • Bi-directed Graphs as a Natural Fit: The use of bi-directed graphs provides an intuitive and powerful way to visualize and define marginal independence structures, which are not easily captured by traditional undirected graphs used in log-linear models.
  • Möbius Parameters Simplify Constraints: The Möbius parameterization is a crucial insight. It transforms complex, non-linear constraints in the original probability space into simpler, linear constraints in the Möbius parameter space, making model fitting and analysis much more tractable.
  • ICF Bridges Theory and Practice: The development of the Iterative Conditional Fitting algorithm for binary data is a significant practical contribution, providing an efficient method to estimate model parameters, which is often a bottleneck in complex graphical models.
  • Symmetry Handling is Crucial: The ability to incorporate symmetry is vital for many real-world datasets, especially those involving paired or repeated observations. This extension makes the proposed models more broadly applicable.

Key Results

  • The paper successfully introduces a new class of models (binary bi-directed graph models) that can represent marginal independence hypotheses, overcoming limitations of traditional log-linear models.
  • An efficient algorithm (ICF) is presented for maximum likelihood estimation in these models.
  • The models are demonstrated on real datasets (twin data and GSS data), showing their applicability and ability to capture relevant dependencies. For the GSS data, a specific marginal independence model was fitted, and its fit was assessed using deviance and chi-squared tests.
  • The paper shows how symmetry constraints can be effectively integrated into these models, leading to more parsimonious and interpretable results, particularly for the twin data.

Limitations and Future Work

  • Computational Complexity: While ICF is efficient, the computational burden can still be significant for very large graphs or high-dimensional data. The paper notes that for larger and denser graphs, alternative approaches might be needed.
  • Positive Counts Assumption: The ICF algorithm is presented assuming positive observed counts. Handling zero counts in maximum likelihood estimation for these models is identified as an open question.
  • Uniqueness of MLE: The uniqueness of the Maximum Likelihood Estimator (MLE) for certain graph structures and count configurations is also mentioned as an open problem.
  • Mixed Continuous and Discrete Variables: Extending these models to handle mixed continuous and discrete variables is noted as a future research direction.
  • Alternative Parametrizations: The paper briefly mentions alternative parametrizations (like dependence ratios) and suggests further exploration of these.

Summary updated

Abstract

Summary Log-linear models are a classical tool for the analysis of contingency tables. In particular, the subclass of graphical log-linear models provides a general framework for modelling conditional independences. However, with the exception of special structures, marginal independence hypotheses cannot be accommodated by these traditional models. Focusing on binary variables, we present a model class that provides a framework for modelling marginal independences in contingency tables. The approach that is taken is graphical and draws on analogies with multivariate Gaussian models for marginal independence. For the graphical model representation we use bidirected graphs, which are in the tradition of path diagrams. We show how the models can be parameterized in a simple fashion, and how maximum likelihood estimation can be performed by using a version of the iterated conditional fitting algorithm. Finally we consider combining these models with symmetry restrictions.
BibTeX
@article{drton2008binary,
  author = {Mathias Drton and Thomas S. Richardson},
  title = {{Binary Models for Marginal Independence}},
  journal = {Journal of the Royal Statistical Society Series B (Statistical Methodology)},
  year = {2008},
  doi = {10.1111/j.1467-9868.2007.00636.x},
}
Figure