Abstract
Scale mixtures of normals have been discussed extensively in the literature as heavy-tailed alternatives to the normal distribution for robust modeling. They have been used either as error models to handle outliers or as prior distributions to provide more reasonable shrinkage of model parameters. The proposed method by Finegold and Drton goes beyond the existing literature both in terms of application (graphical models) and methodology (Dirichlet $t$) for outlier handling. While this approach can be applied to many other problems, in this discussion I will focus on its application in Bayesian modeling of high throughput biological data.
Introduction
In high-dimensional biological data—encompassing thousands of genes or millions of genetic markers—the standard Normal distribution often proves inadequate. Its thin tails treat significant deviations as impossible outliers, struggling to distinguish genuine biological signals from random noise. In a commentary on the work of Finegold and Drton (2014), statistician Babak Shahbaba explores how the Dirichlet $t$-distribution can revolutionize the modeling of complex biological systems.
The Challenge of High-Dimensionality
High-throughput biology frequently encounters the "p >> n" problem, where the number of variables (e.g., genes) vastly exceeds the number of samples. To navigate this, researchers employ prior distributions to "shrink" the effects of most variables toward zero, assuming only a subset are biologically relevant.
The standard Gaussian model is "light-tailed," assigning extremely low probability to large values. Consequently, a Normal prior may excessively shrink a true, large biological effect, effectively silencing a discovery. Furthermore, real-world data is often noisy; without a robust model, outliers can significantly skew results.
Methodology: Scale Mixtures and Dirichlet Processes
The core innovation lies in Scale Mixtures of Normals. Instead of a fixed bell curve, a variable is modeled as a Normal distribution where the variance (the "scale") is itself a random variable.
- Student’s $t$ as a Mixture: Assuming the variance follows an Inverse-Gamma distribution results in a Student’s $t$ marginal distribution. Its heavier tails offer greater robustness than the Normal distribution.
- The Dirichlet Process: Rather than assuming independent variances for every variable, a Dirichlet Process allows the model to cluster variables sharing similar scales.
- The Dirichlet $t$: This generalizes the $t$-distribution by using the Dirichlet process to govern scale parameters, providing nonparametric flexibility.
- The Dirichlet Horseshoe: Shahbaba suggests extending this by using a half-Cauchy distribution as the base for the Dirichlet process. This creates a "Horseshoe" effect—extremely heavy tails that enforce sparsity (keeping zero-effects at zero) while preserving large, true signals.
Key Insights and Results
Distributions like the $t$, Laplace, and Horseshoe are superior for high-dimensional priors because they feature a "tall peak" at zero to capture noise and "flat tails" to accommodate large signals. By using a Dirichlet process, the model automatically learns groups of genes with similar variability.
As shown in the figure above, the Dirichlet Horseshoe creates a distinct "cross" shape compared to the circular cluster of the standard Normal distribution. This shape represents the model's ability to strictly enforce sparsity (the vertical and horizontal axes) while allowing for significant outliers (points far from the center). In Genome-Wide Association Studies (GWAS), such "alternative $t$-distribution" specifications have been shown to outperform standard Normal and Student’s $t$ models by better handling the shrinkage of Single Nucleotide Polymorphisms (SNPs).
Conclusion and Future Directions
While these models demonstrate empirical success in identifying differentially expressed genes and handling GWAS data, a complete theoretical framework is still developing. The mathematical properties of the Dirichlet $t$ and Dirichlet Horseshoe require further mapping. Future research must provide a rigorous theoretical basis for their performance in high-dimensional settings and explore the use of non-conjugate distributions, such as the half-Cauchy, within these Bayesian hierarchies.
Summary updated