Robust Bayesian inference of networks using Dirichlet t-distributions

Michael Finegold, Mathias DrtonarXiv 2012 · DOI · arXiv

Abstract

Bayesian graphical modeling offers a powerful framework for inferring network structures while accounting for uncertainty. While Gaussian models are the standard choice, they are highly sensitive to outliers—a common issue in high-dimensional biological data like gene expression profiles. To address this, we propose the Dirichlet $t$-distribution, a flexible extension of the multivariate $t$-distribution. By using a Dirichlet Process to adaptively cluster latent scaling factors within a single observation, this model interpolates between the rigid classical $t$-distribution and the computationally demanding alternative $t$-distribution. Our approach provides robust network inference, handles "pockets of contamination" within data points, and maintains computational efficiency in high-dimensional settings.

Introduction

In systems biology and genomics, researchers frequently use Gaussian Graphical Models (GGMs) to map dependencies between variables, such as gene-to-gene interactions. However, real-world data is rarely perfectly Gaussian. Technical glitches or localized biological anomalies can introduce outliers that skew the entire inferred network.

Standard robust alternatives, such as the multivariate $t$-distribution, typically rely on a latent Gamma random variable to rescale a Gaussian vector. However, existing implementations suffer from a "Goldilocks problem":

  1. The Classical $t$-distribution uses a single scaling factor for the entire observation. If one variable is an outlier, the model treats the whole experiment as an outlier, discarding potentially useful data.
  2. The Alternative $t$-distribution assigns an independent scaling factor to every variable. While flexible, this increases computational complexity and imposes artificial constraints on the achievable correlation structures between variables.

The Dirichlet $t$-Distribution

We propose a middle ground: the Dirichlet $t$-distribution. This model represents the $t$-distribution as a latent Gaussian vector divided by a scaling factor (a Gamma divisor), but introduces a Dirichlet Process (DP) to cluster these divisors.

Adaptive Clustering Within Observations

Unlike typical applications of Dirichlet Processes that cluster different observations, our model uses the DP to cluster variables within a single multivariate observation. This allows the model to automatically identify which variables are "clean" (grouping them into a shared cluster with a standard divisor) and which are "contaminated" (assigning them to unique clusters with heavy-tail divisors).

Interpolation and Flexibility

By adjusting the DP concentration parameter $\alpha$, the Dirichlet $t$-distribution naturally interpolates between the two extremes:

  • As $\alpha \to 0$, the model collapses to the Classical $t$.
  • As $\alpha \to \infty$, the model becomes the Alternative $t$.

This flexibility allows the model to handle "pockets of contamination"—where only a subset of measurements in an experiment are outliers—without the computational explosion or correlation bounds associated with the Alternative $t$ model.

Methodology

We developed a Markov Chain Monte Carlo (MCMC) algorithm to perform inference. The sampler combines Metropolis-Hastings moves to explore the space of decomposable graph structures with Gibbs sampling for the latent cluster assignments and Gamma divisors. This approach ensures that the model can adaptively learn the outlier structure of the data while simultaneously inferring the underlying network.

Experimental Results

Simulation Studies

We tested the model using simulated data, including an AR1 process (chain graph) and random graphs with $p=100$ nodes. We manually injected contamination into submatrices of the data to simulate localized outliers.

  • Robustness: The Dirichlet $t$ consistently outperformed Normal and Classical $t$ models in Precision-Recall and ROC metrics.
  • Efficiency: In high-dimensional settings ($p=100$), the Dirichlet model reached equilibrium significantly faster than the Alternative $t$ model, which often struggled to converge.

Real-World Application: Yeast Microarray Data

We applied the model to a well-known yeast microarray dataset (Gasch et al., 2000), focusing on genes involved in galactose utilization. The Dirichlet $t$ model successfully identified specific experiments where only a subset of genes behaved as outliers. By "downweighting" only these specific messy values rather than the entire experiment, the model extracted a more accurate network structure than classical methods.

Discussion and Future Work

Our Dirichlet $t$-distribution provides a robust, computationally feasible framework for Bayesian network inference. By allowing for adaptive clustering of outliers, it preserves the integrity of high-dimensional datasets that are partially contaminated.

Current limitations include the restriction to decomposable graphs to maintain mathematical tractability with Hyper Inverse Wishart priors. Future research may explore extending this to non-decomposable structures or developing "doubly Dirichlet" models that share outlier patterns across different observations to further enhance statistical power.

Summary updated

Abstract

Bayesian graphical modeling provides an appealing way to obtain uncertainty estimates when inferring network structures, and much recent progress has been made for Gaussian models. These models have been used extensively in applications to gene expression data, even in cases where there appears to be significant deviations from the Gaussian model. For more robust inferences, it is natural to consider extensions to t-distribution models. We argue that the classical multivariate t-distribution, defined using a single latent Gamma random variable to rescale a Gaussian random vector, is of little use in highly multivariate settings, and propose other, more flexible t-distributions. Using an independent Gamma-divisor for each component of the random vector defines what we term the alternative t-distribution. The associated model allows one to extract information from highly multivariate data even when most experiments contain outliers for some of their measurements. However, the use of this alternative model comes at increased computational cost and imposes constraints on the achievable correlation structures, raising the need for a compromise between the classical and alternative models. To this end we propose the use of Dirichlet processes for adaptive clustering of the latent Gamma-scalars, each of which may then divide a group of latent Gaussian variables. Dirichlet processes are commonly used to cluster independent observations; here they are used instead to cluster the dependent components of a single observation. The resulting Dirichlet t-distribution interpolates naturally between the two extreme cases of the classical and alternative t-distributions and combines more appealing modeling of the multivariate dependence structure with favorable computational properties.
BibTeX
@misc{finegold2012robust,
  author = {Michael Finegold and Mathias Drton},
  title = {{Robust Bayesian inference of networks using Dirichlet t-distributions}},
  year = {2012},
  eprint = {1207.1221},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1207.1221},
}
Figure