Model Uncertainty in Latent Gaussian Models with Univariate Link Function

Mark F. J. Steel, Gregor Zens

Abstract

We consider a class of latent Gaussian models with a univariate link function (ULLGMs). These are based on standard likelihood specifications (such as Poisson, Binomial, Bernoulli, Erlang, etc.) but incorporate a latent normal linear regression framework on a transformation of a key scalar parameter. We allow for model uncertainty regarding the covariates included in the regression. The ULLGM class typically accommodates extra dispersion in the data and has clear advantages for deriving theoretical properties and designing computational procedures. We formally characterize posterior existence under a convenient and popular improper prior and show that ULLGMs inherit the consistency properties from the latent Gaussian model. We propose a simple and general Markov chain Monte Carlo algorithm for Bayesian model averaging in ULLGMs. Simulation results suggest that the framework provides accurate results that are robust to some degree of misspecification. The methodology is successfully applied to measles vaccination coverage data from Ethiopia and to data on bilateral migration flows between OECD countries.

Introduction

In regression analysis, Gaussian (Normal) models are often favored for their mathematical tractability and well-understood properties. However, real-world data frequently manifests as counts, proportions, or rates, necessitating non-Gaussian likelihoods like Poisson or Binomial distributions. Standard Generalized Linear Models (GLMs) for these data types face two significant challenges: overdispersion and model uncertainty.

Overdispersion occurs when the observed variance in the data exceeds what the theoretical model predicts—a common occurrence in empirical research. This leads to overconfident predictions and biased variable selection. Furthermore, when faced with a large set of potential predictors, researchers encounter model uncertainty. While Bayesian Model Averaging (BMA) offers an elegant solution in Gaussian settings, the non-linear nature of GLMs makes the required integrals analytically intractable, often forcing a choice between rough approximations or computationally expensive algorithms.

The ULLGM Framework

To address these issues, this paper introduces Univariate Link Latent Gaussian Models (ULLGMs). This framework inserts a hidden Gaussian layer between the predictors and the outcome, creating a three-layer hierarchy:

  1. The Observation Layer: The data $y$ follows a standard distribution $F$, where a key parameter (e.g., the mean $\lambda$ in a Poisson model) is controlled by a latent variable $z$.
  2. The Latent Layer: The variable $z$ follows a linear regression model: $z = \alpha + X\beta + \epsilon$, where $\epsilon \sim N(0, \sigma^2)$.
  3. The Link Function: A transformation $h(z)$ maps the latent Gaussian space to the parameter space of the observation distribution.
Table 1: Summary of various ULLGM specifications, including their observation spaces, likelihoods, and link functions.

By conditioning on the latent $z$, the parameters $\alpha, \beta,$ and $\sigma^2$ can be integrated out analytically. This "Data Augmentation" strategy allows the use of efficient Gaussian BMA machinery, such as $g$-priors and Gibbs sampling, even for non-Gaussian outcomes.

Modeling Overdispersion

A key advantage of ULLGMs is their natural handling of overdispersion. The error term in the latent layer ($\sigma^2$) acts as a "pressure valve." Instead of forcing the model to select additional, potentially spurious variables to explain high variance, the latent noise term absorbs the extra dispersion.

Figure 1: Density plots for Poisson Log-Normal and Binomial Logistic Normal models. As the latent variance $\sigma^2$ increases, the distributions become more dispersed, demonstrating the model's flexibility in handling overdispersion.

As shown in Figure 1, increasing $\sigma^2$ allows the model to capture a wider range of data behaviors while maintaining the underlying structure of the Poisson or Binomial process.

Empirical Performance and Insights

The authors validated the ULLGM framework through extensive simulations and two real-world applications:

  • Ethiopian Measles Vaccination: Modeling vaccination rates across 305 clusters using 63 potential predictors.
  • OECD Migration Flows: Analyzing 1,406 bilateral migration flows between 38 countries with 54 variables. This dataset is extremely overdispersed, with variance exceeding the mean by a factor of 345,000.

Key Results

  • Superior Prediction: ULLGMs consistently outperformed standard GLMs in terms of Log Predictive Scores (LPS), indicating better generalization to held-out data.
  • Smarter Variable Selection: While standard models often selected overly complex models to compensate for unexplained variance, ULLGMs identified sparser, more interpretable sets of predictors.
  • Computational Efficiency: The MCMC algorithm proved highly efficient; for instance, the migration data analysis took only 30 minutes for 300,000 iterations on a standard laptop.
  • Theoretical Soundness: The paper proves that ULLGMs inherit "model selection consistency" from Gaussian models, ensuring that the method reliably identifies the true underlying model as sample size increases.

Conclusion

ULLGMs provide a robust and computationally efficient bridge between the flexibility of non-Gaussian likelihoods and the analytical power of Gaussian BMA. Future research may focus on formally testing the necessity of the latent layer (the boundary problem of $\sigma^2 = 0$), extending the framework to multivariate outcomes, and further optimizing scaling for high-dimensional datasets with thousands of predictors.

Summary updated

Abstract

We consider a class of latent Gaussian models with a univariate link function (ULLGMs). These are based on standard likelihood specifications (such as Poisson, Binomial, Bernoulli, Erlang, etc.) but incorporate a latent normal linear regression framework on a transformation of a key scalar parameter. We allow for model uncertainty regarding the covariates included in the regression. The ULLGM class typically accommodates extra dispersion in the data and has clear advantages for deriving theoretical properties and designing computational procedures. We formally characterize posterior existence under a convenient and popular improper prior and show that ULLGMs inherit the consistency properties from the latent Gaussian model. We propose a simple and general Markov chain Monte Carlo algorithm for Bayesian model averaging in ULLGMs. Simulation results suggest that the framework provides accurate results that are robust to some degree of misspecification. The methodology is successfully applied to measles vaccination coverage data from Ethiopia and to data on bilateral migration flows between OECD countries.
BibTeX
@inproceedings{steel2025model,
  author = {Mark F. J. Steel and Gregor Zens},
  title = {{Model Uncertainty in Latent Gaussian Models with Univariate Link Function}},
  booktitle = {Bayesian},
  year = {2025},
  doi = {10.1214/25-ba1538},
}
Figure