TL;DR
This paper introduces a novel statistical learning procedure for discovering the structure of graphical models that combine both discrete and continuous variables, particularly in scenarios where the number of variables (p) far exceeds the number of observations (n). By adapting a limited-order correlation approach, the authors provide a method that effectively handles complex, mixed-type data, a common challenge in fields like genomics.
What Problem Does This Paper Address?
Traditional graphical model learning methods often assume that the number of observations (n) is significantly larger than the number of variables (p). However, modern scientific endeavors, especially in areas like genomics and high-throughput experiments, frequently generate datasets where p >> n. While methods for learning Gaussian graphical models in this high-dimensional setting exist (e.g., using regularization or dimension reduction), the problem becomes considerably more challenging when dealing with mixed discrete and continuous variables. Existing approaches often rely on linear modeling outside the graphical model framework or are restricted to decomposable models. This paper aims to fill this gap by developing a method for learning the structure of non-decomposable mixed graphical models in the p >> n setting.
Methods
The core of the paper's contribution lies in adapting a "limited-order correlation" approach, previously developed for Gaussian graphical models by Castelo and Roverato (2006), to handle mixed discrete and continuous variables. The fundamental idea is to estimate associations between pairs of variables by considering their marginal distributions of a limited order (q+2).
Key methodological aspects include:
- Mixed Graphical Markov Models (GMMs): The paper defines and utilizes GMMs for distributions involving both discrete (denoted by $\mathcal{I}$) and continuous (denoted by $\mathcal{Y}$) random variables, represented by a marked graph $G = (V, E)$.
- Limited-Order Correlations: Instead of direct pairwise associations, the method estimates associations by testing conditional independence relationships. This involves sampling subsets of variables ($Q$) and testing $X_\alpha \perp X_\beta | X_Q$.
- Exact Conditional Independence Tests: To address the p >> n challenge and small sample sizes, the authors derive exact conditional independence tests based on likelihood ratios. These tests are shown to provide better control of the significance level compared to asymptotic tests, especially for smaller sample sizes. These tests are derived for both missing continuous edges and missing mixed edges.
- Beta Distribution for Likelihood Ratios: For both pure continuous and mixed variable cases, the likelihood ratio statistics are shown to follow a beta distribution, allowing for exact hypothesis testing.
- Homogeneous Mixed GMM Assumption: For simplification and to meet sample size requirements for maximum likelihood estimates, the method assumes homogeneous mixed GMMs.
- Marginally Independent Discrete Variables: A further simplifying assumption is made that discrete variables are marginally independent.
Datasets
The paper uses two types of datasets:
- Synthetic Data: The authors generate synthetic data from homogeneous mixed GMMs. They control graph density (using d-regular graphs) and the strength of correlations (Pearson correlation $\rho$ and standard deviation $\sigma$ for mixed interactions). This allows for systematic evaluation of their method's performance under controlled conditions.
- Real Data: The authors apply their method to a real dataset from a yeast gene expression and genotype study (Brem and Kruglyak, 2005). This dataset consists of 112 samples with 6,216 genes and 2,906 genotype markers.
Key Insights and Ideas
- Exact Tests for Small Samples: The development and validation of exact conditional independence tests are crucial. In the p >> n setting, sample sizes can be small, making asymptotic tests unreliable. The exact beta-distribution-based tests offer superior control over Type-I errors.
- Bridging the Gap for Mixed Data: The paper successfully extends a powerful technique (limited-order correlations) from the well-studied Gaussian GMM domain to the more complex mixed-variable domain, a significant step for analyzing heterogeneous biological data.
- Performance in High Dimensions: The study demonstrates that even with p >> n, the limited-order correlation approach, when combined with exact tests, can effectively learn graph structures, although performance naturally degrades as graph complexity increases.
- Decomposability vs. Non-Decomposability: By focusing on non-decomposable models, the paper addresses a broader class of graphical models than some previous work that was restricted to decomposable structures.
Key Results
- Type-I Error Control: Simulations show that the exact conditional independence tests provide better control of the Type-I error rate compared to asymptotic tests, especially as sample size decreases and graph density increases. The exact test kept the error rate approximately constant.
- Performance Comparison: When compared to the gRapHD method (which is restricted to decomposable models), the proposed "qpgraph" method shows competitive performance, particularly in terms of Area Under the Curve (AUC) on precision-recall curves. The paper notes that gRapHD's performance degrades more dramatically when graph complexity increases, likely due to its decomposability constraint.
- Real Data Application: The analysis on yeast data suggests that the method can help identify more direct associations between genetic variation and gene expression, potentially distinguishing between direct and indirect effects more effectively than univariate approaches.
Limitations and Future Work
- Homogeneity Assumption: The method assumes homogeneous mixed GMMs. Relaxing this assumption to handle heterogeneous models would increase its applicability.
- Marginal Independence of Discrete Variables: The assumption that discrete variables are marginally independent simplifies the problem but might not hold in all real-world scenarios. Future work could explore methods that relax this constraint.
- Computational Complexity: While the method is designed for p >> n, the computational cost of estimating correlations and performing tests can still be significant, especially for very large p. Further optimizations might be needed.
- Choice of q: The method relies on a parameter 'q' for the order of correlations. While the paper suggests averaging over different 'q' values or using a fixed small 'q' (like 3), finding an optimal 'q' or a data-driven way to select it could be an area for future research.
- Implementation: The paper mentions that the methodology is implemented in the R package
qpNrr()withinqpgraph, making it available for further use and development.
Summary updated