Abstract
We consider the problem of learning the structure of a pairwise graphical model over continuous and discrete variables. We present a new pairwise model for graphical models with both continuous and discrete variables, utilizing a group-lasso penalty to a joint pseudolikelihood objective to learn the underlying structure of complex, mixed-type data.
Introduction
In many real-world applications, datasets are composed of heterogeneous variables. For instance, a medical study might record continuous measurements like blood pressure alongside discrete categorical data like blood type. Representing the conditional dependencies between these variables is a fundamental task in statistics, typically handled via graphical models. While established methods exist for purely continuous data (Gaussian Graphical Models) or purely discrete data (Ising or Potts models), "mixed" models have historically been computationally expensive or required restrictive assumptions. This paper introduces a unified pairwise graphical model designed to bridge this gap, offering a flexible and scalable approach for high-dimensional settings.
Methodology
The proposed Pairwise Mixed Graphical Model defines a joint probability density where interactions between any two variables are captured by potential functions. The model is constructed to ensure conditional consistency: the conditional distribution of a continuous variable given all others follows a Gaussian linear regression, while the conditional distribution of a discrete variable follows a multiclass logistic regression.
To estimate the model parameters in high-dimensional spaces, the authors employ Pseudolikelihood Estimation. This approach avoids the intractable normalization constant of the full joint likelihood by optimizing the product of conditional distributions, maintaining statistical consistency while significantly reducing computational overhead.
Structure learning is achieved through Group-Lasso Regularization. To account for the varying dimensions of parameters (e.g., scalars for continuous-continuous edges vs. matrices for discrete-discrete edges), a calibrated weighting scheme is introduced. This ensures that the penalty is balanced across different edge types. The optimization problem is solved using Proximal Gradient and Proximal Newton methods, which efficiently handle the combination of smooth likelihood functions and non-smooth penalties.
Experimental Evaluation
The model was evaluated using both synthetic and real-world datasets. Synthetic tests involved 10 continuous and 10 binary variables to assess the algorithm's ability to recover a known ground-truth structure.
As shown in Figure 1, the model exhibits a sharp phase transition in recovery performance. With a sufficient sample size (approximately $n=1000$ for 20 variables), the probability of exact structure recovery reaches nearly 100%.
The model was also applied to the March 2011 Census Population Survey dataset, which includes variables such as age, log-wage, education, and health insurance status.
Results on the Census data (Figure 2) indicate that the joint pseudolikelihood approach outperforms separate node-wise regressions, particularly at smaller sample sizes. By enforcing symmetry and global regularization, the joint model provides more stable estimates of relationships between socio-economic variables.
Discussion and Limitations
A key insight of this work is the importance of the calibrated regularizer, which prevents bias against discrete-discrete edges that naturally involve more parameters. Furthermore, the model serves as a natural generalization: it reduces to the standard Graphical Lasso or a pairwise Markov Random Field when only one type of variable is present.
However, some limitations remain. The model assumes only pairwise interactions, potentially overlooking higher-order dependencies. Additionally, while pseudolikelihood is efficient, the Proximal Newton method's reliance on Hessian information may limit scaling to extremely high-dimensional datasets. Finally, the assumption of Gaussian-like behavior for continuous variables may not hold for highly skewed data, suggesting a need for future non-parametric extensions.
Summary updated