Quantile Graphical Models: a Bayesian Approach

Nilabja Guha, Veerabhadran Baladandayuthapani, Bani K. MallickJMLR 2020

TL;DR

This paper introduces "Quantile Graphical Models" (QGMs), a novel Bayesian approach to inferring relationships between variables that is robust to non-Gaussian data and outliers. By modeling conditional quantiles instead of means, QGMs offer a more flexible and reliable way to uncover complex dependencies, especially in real-world datasets with unusual distributions.

What Problem Does This Paper Address?

Traditional graphical models, like Gaussian Graphical Models (GGMs), often assume data follows a multivariate normal distribution. This assumption is frequently violated in practice, leading to inaccurate inference of variable relationships. When data deviates from normality, GGMs can incorrectly identify connections or miss existing ones, especially in the presence of outliers or heavy-tailed distributions. This limits their applicability in fields like genomics, finance, and social sciences where such data characteristics are common. The paper aims to overcome these limitations by developing a graphical model that is robust to distributional assumptions and can effectively capture complex dependencies.

Methods

The core innovation is the use of quantile regression within a Bayesian framework to model the conditional relationships between variables. Instead of focusing on the conditional mean (as in GGMs), QGMs model the conditional quantiles. The paper proposes that if a variable's conditional quantile function is linear and independent of another variable across all quantiles, then those variables are conditionally independent. This transforms the graphical model problem into a variable selection problem within quantile regression.

Key methodological contributions include:

  • Bayesian Quantile Graphical Models: A hierarchical Bayesian framework is developed to model conditional quantile functions.
  • Spike-and-Slab Priors: These are used for variable selection in the quantile regression setup, allowing for sparse graph estimation.
  • Neighborhood Selection: The approach focuses on identifying the neighbors of each node by performing variable selection on quantile regression coefficients.
  • Variational Bayes Approximation: To handle large datasets, an efficient variational Bayes algorithm is proposed for approximating the posterior distribution, offering a faster alternative to MCMC.
  • Consistency Proofs: The paper provides theoretical guarantees for the consistency of the proposed graph estimation method under both true and misspecified models.

Datasets

The paper evaluates its method on:

  • Simulated Datasets: Several simulated examples are used to demonstrate the method's performance under various scenarios, including non-Gaussian distributions, heavy tails, and sparse structures.
  • Cancer Genome Atlas (TCGA) Lung Squamous Cell Carcinoma (LUSC) Data: This real-world dataset, containing proteomic profiles from over 8000 patient tumors, is used to construct a protein signaling network. This dataset is particularly relevant as it often exhibits complex, non-Gaussian relationships.

Key Insights and Ideas

  • Robustness to Non-Normality: By shifting focus from the mean to quantiles, QGMs are inherently more robust to deviations from Gaussianity and the presence of outliers. This is a significant advantage over traditional GGMs.
  • Capturing Complex Dependencies: Quantile regression can capture more nuanced relationships than linear regression, allowing QGMs to uncover dependencies that might be missed by mean-based methods, especially in the tails of distributions.
  • Bayesian Uncertainty Quantification: The Bayesian approach provides a principled way to quantify uncertainty in the estimated graph structure, which is crucial for reliable inference.
  • Scalability for Large Datasets: The development of a variational Bayes approximation is key to making this powerful method applicable to modern, high-dimensional datasets.

Key Results

  • Improved Performance on Non-Gaussian Data: Simulations show that QGMs significantly outperform GGMs (both GLASSO and MB implementations) in recovering true graph structures when data is non-Gaussian, exhibiting fewer false positives and better separation of independent components.
  • Accurate Protein Network Construction: The QGM method successfully identified known connections in the LUSC proteomic data and also discovered novel connections that warrant further biological investigation.
  • Computational Efficiency: The variational Bayes algorithm is demonstrated to be orders of magnitude faster than MCMC methods, making it practical for large-scale applications.
  • Theoretical Guarantees: The paper provides theoretical proofs for the consistency of the proposed method, ensuring that the estimated graph converges to the true graph under certain conditions.

Limitations and Future Work

  • Model Misspecification: While the paper proves consistency under misspecification, the performance can still be affected by the degree of misspecification.
  • Computational Cost of MCMC: Although variational Bayes is proposed, the MCMC approach, while more computationally intensive, is still presented as a viable option.
  • Linearity Assumption: The current model assumes linearity of the conditional quantile functions. Relaxing this assumption to accommodate more general functional forms could further enhance the model's applicability.
  • Further Investigation of Model Selection Properties: The authors suggest future work to delve deeper into the model selection properties for each node and their related convergence rates.
  • Validation of New Discoveries: The newly discovered connections in the protein network require further biological validation.

Summary updated

BibTeX
@article{guha2020quantile,
  author = {Nilabja Guha and Veerabhadran Baladandayuthapani and Bani K. Mallick},
  title = {{Quantile Graphical Models: a Bayesian Approach}},
  journal = {JMLR},
  year = {2020},
}
Figure