Hidden Gibbs random fields model selection using Block Likelihood Information Criterion

Julien Stoehr, Jean‐Michel Marin, Pierre PudloStat 2016 · DOI · arXiv

TL;DR

This paper introduces the Block Likelihood Information Criterion (BLIC), a novel method for selecting models of hidden Gibbs random fields. BLIC overcomes the computational intractability of traditional methods by approximating the likelihood using block-wise distributions, offering a more efficient and reliable alternative for complex spatial data analysis.

What Problem Does This Paper Address?

Hidden Gibbs random fields are powerful tools for modeling spatially correlated data, such as in image analysis or disease mapping. However, a major hurdle in using these models is the computational difficulty in calculating the normalizing constant (partition function) of the probability distribution. This intractability makes it challenging to evaluate the likelihood of the data given a model, which is crucial for tasks like model selection – choosing the best model from a set of candidates. Traditional methods like the Bayesian Information Criterion (BIC) rely on this likelihood, but their direct application to hidden Gibbs random fields is often impossible. This paper tackles this "doubly intractable" problem by proposing a new criterion that can efficiently perform model selection, specifically for determining the number of latent states and the dependency structure within these fields.

Methods

The core innovation of this paper is the Block Likelihood Information Criterion (BLIC). The authors propose to approximate the intractable Gibbs distribution by partitioning the spatial lattice into contiguous rectangular blocks. Within each block, they approximate the probability measure by a product of Gibbs distributions. This "block-wise" approximation allows for a tractable estimation of the likelihood.

Specifically, BLIC is derived by replacing the full Gibbs likelihood with a product of distributions over these blocks. This approach leverages the idea of composite likelihoods but crucially maintains a valid probability distribution, avoiding potential misspecification issues. The authors also discuss how to compute the necessary components of BLIC, drawing on existing recursive algorithms for efficient calculation. They compare BLIC to existing criteria like the Pseudolikelihood Information Criterion (PLIC) and mean field-like approximations, as well as to Approximate Bayesian Computation (ABC) methods.

Datasets

The paper does not introduce new datasets. Instead, it uses simulated data generated from hidden Potts models to evaluate the performance of BLIC. These simulations are designed to mimic real-world scenarios where spatial dependencies and latent states are present.

Key Insights and Ideas

  • Bridging the Intractability Gap: BLIC offers a practical solution to the long-standing problem of model selection for hidden Gibbs random fields, which is plagued by computationally intractable likelihoods.
  • The Power of Blocking: Partitioning the spatial lattice into blocks and approximating the distribution within these blocks is a key strategy that enables efficient likelihood estimation.
  • Trade-off Between Efficiency and Accuracy: BLIC provides a compelling balance between computational speed and the reliability of model selection results, outperforming some existing methods.
  • Importance of Block Size: The performance of BLIC is sensitive to the size of the blocks used. Larger blocks can be more informative for selecting dependency structures, but may also increase computational cost.
  • Beyond Independent Models: Unlike simpler criteria that assume independence, BLIC accounts for spatial dependencies, leading to better performance in many scenarios.

Key Results

The paper demonstrates that BLIC generally performs well in selecting both the number of latent states and the dependency structure of hidden Potts models.

  • Number of Colors Selection: In experiments focused on selecting the number of latent states (colors), BLIC variants (e.g., BLIC$_{2\times2}$, BLIC$_{4\times4}$) consistently outperformed criteria like PLIC and mean field-like approximations, especially when the spatial dependency parameter was set near the phase transition.
  • Dependency Structure Selection: For selecting the dependency structure, BLIC$_{2\times2}$ showed promise, though the paper notes that larger blocks are generally needed for better discrimination between complex neighborhood structures.
  • Comparison with ABC: BLIC offers a competitive alternative to ABC methods, providing comparable or better error rates at a significantly lower computational cost, making it more practical for many applications.

Limitations and Future Work

The authors acknowledge several limitations and suggest avenues for future research:

  • Optimal Block Construction: The current method for partitioning the lattice into blocks is based on simple rectangular shapes and relies on optimality conditions that are yet to be fully explored. Future work could investigate more sophisticated block constructions.
  • Penalty Term Refinement: The penalty term in BLIC is currently based on the assumption of independent variables within blocks. This simplification might slightly alter the effective number of free parameters. Future research could explore how intra-block dependencies impact the penalty term and how to refine it for greater accuracy.
  • Generalization to Other Models: While tested on hidden Potts models, exploring the applicability and performance of BLIC on other types of hidden Gibbs random fields would be valuable.

Summary updated

Abstract

Performing model selection between Gibbs random fields is a very challenging task. Indeed, because of the Markovian dependence structure, the normalizing constant of the fields cannot be computed using standard analytical or numerical methods. Furthermore, such unobserved fields cannot be integrated out, and the likelihood evaluation is a doubly intractable problem. This forms a central issue to pick the model that best fits an observed data. We introduce a new approximate version of the Bayesian Information Criterion . We partition the lattice into contiguous rectangular blocks, and we approximate the probability measure of the hidden Gibbs field by the product of some Gibbs distributions over the blocks. On that basis, we estimate the likelihood and derive the Block Likelihood Information Criterion (BLIC) that answers model choice questions such as the selection of the dependence structure or the number of latent states. We study the performances of BLIC for those questions. In addition, we present a comparison with ABC algorithms to point out that the novel criterion offers a better trade‐off between time efficiency and reliable results. Copyright © 2016 John Wiley & Sons, Ltd.
BibTeX
@inproceedings{stoehr2016hidden,
  author = {Julien Stoehr and Jean‐Michel Marin and Pierre Pudlo},
  title = {{Hidden Gibbs random fields model selection using Block Likelihood Information Criterion}},
  booktitle = {Stat},
  year = {2016},
  doi = {10.1002/sta4.112},
}
Figure