Modelling Nonstationary Gene Regulatory Processes

Marco Grzegorcyzk, Dirk Husmeier, Jörg RahnenführerAdvances 2010 · DOI

TL;DR

This paper introduces the BGMD model, a novel approach for inferring gene regulatory networks from dynamic gene expression data. By combining a changepoint process with Bayesian networks, BGMD effectively models non-stationary biological processes, outperforming existing methods like the BGM model in accuracy and robustness across various biological systems.

What Problem Does This Paper Address?

Understanding how genes regulate each other over time is crucial for deciphering complex biological systems. Gene expression data, collected as time series, offers a window into these dynamic processes. However, traditional methods for inferring these gene regulatory networks, such as static Bayesian networks or dynamic Bayesian networks (DBNs) with linear models, struggle with the inherent non-linearity and changing nature (non-stationarity) of gene regulation. While some methods attempt to address this by allowing for flexible model structures or data partitioning, they often suffer from theoretical limitations, computational complexity, or a tendency to overfit. This paper tackles the challenge of accurately modeling and inferring these dynamic, non-linear gene regulatory networks, particularly when the underlying regulatory mechanisms change over time.

Methods

The core contribution of this paper is the BGMD (Bayesian Gaussian Mixture with Changepoints) model. It builds upon the Bayesian Gaussian Mixture (BGM) model, which uses a "free allocation" approach to partition time series data into different "compartments" or segments, allowing for different regulatory parameters in each. However, the authors identify theoretical shortcomings in the free allocation method.

BGMD replaces the free allocation with a multiple changepoint process. This means that instead of allowing any data point to be assigned to any compartment independently, BGMD assumes that adjacent time points are more likely to belong to the same regulatory state. This is achieved by inferring the locations of changepoints that divide the time series into segments.

Key methodological aspects include:

  • Dynamic Bayesian Networks (DBNs): The framework for modeling time-dependent relationships between genes.
  • Gaussian BGe Score: A standard method for scoring network structures with continuous data, assuming Gaussian distributions.
  • Changepoint Process: The novel component that divides the time series into segments based on inferred changepoint locations. This is implemented using a reversible jump Markov chain Monte Carlo (RJMCMC) algorithm, allowing the model to infer the number and locations of changepoints.
  • MCMC Inference: A computational approach to sample from the posterior distribution of network structures and changepoint configurations.
  • Discrete BDM/BDMD Models: The paper also extends the BGMD concept to discrete data by replacing the Gaussian BGe score with the discrete multinomial BDe score, creating the Bayesian Discrete Mixture (BDM) and Bayesian Discrete Mixture with Changepoints (BDMD) models.

Datasets

The paper evaluates its proposed BGMD model (and its discrete counterpart, BDMD) using a combination of synthetic and real biological datasets:

  • Synthetic Datasets:
    • Sinusoidally generated time series: Used to test the model's ability to infer networks with known ground truth, particularly in the presence of non-linearity and noise. These datasets were generated with varying levels of noise (cx and cy) and different numbers of nodes (N=2 and N=4).
  • Biological Datasets:
    • Macrophage gene expression time series: Data from macrophages infected with Cytomegalovirus (CMV) or treated with Interferon Gamma (IFNy), or both. This dataset was used to study host response to infection and immune activation.
  • Arabidopsis thaliana gene expression time series: Data from plants subjected to different light-dark cycle entrainments, used to study circadian regulation.
  • Drosophila melanogaster gene expression time series: Binary data representing muscle development stages, used to compare the free allocation and changepoint models for discrete data.

Key Insights and Ideas

  • Changepoints are more natural for time series: The core insight is that for dynamic biological processes, assuming adjacent time points belong to the same regulatory state (via changepoints) is more biologically plausible and statistically robust than allowing arbitrary free allocation of data points to segments.
  • BGMD suppresses spurious self-feedback loops: The free allocation model (BGM) can be prone to inferring false self-feedback loops, especially in non-stationary scenarios. BGMD, by enforcing temporal structure, significantly reduces this problem.
  • Prior structure matters: The paper highlights how the prior probability distributions inherent in the models (specifically, the prior on segment allocation) strongly influence their performance. BGMD's prior favors more balanced segment lengths, which aligns better with many biological processes.
  • BGM's penalty for symmetry: The BGM model's prior heavily penalizes symmetric segmentations, leading it to favor very short segments or outliers, which can be artifacts rather than true biological signals, especially in longer time series.
  • BGMD offers better generalization: The BGMD model consistently shows better predictive probabilities, indicating it generalizes better to unseen data and is less prone to overfitting compared to BGM.

Key Results

  • Synthetic Data: BGMD consistently achieved higher Area Under the ROC (AUC) scores for network reconstruction accuracy compared to BGe and BGM across various noise levels and network sizes.
  • Macrophages Data: BGMD inferred a clearer two-stage regulatory process, reflecting the distinct responses to infection and immune activation, compared to BGM. Predictive probabilities for BGMD were systematically better.
  • Arabidopsis thaliana Data: BGMD identified a stronger separation between the two circadian rhythm stages and provided better predictive probabilities than BGM. The reconstructed network also showed biologically plausible interactions.
  • Drosophila melanogaster Data: The discrete BDMD model inferred a segmentation that better corresponded to the morphological stages of development compared to the BDM model, which tended to infer fragmented segments.
  • Theoretical Analysis: Prior analysis showed that BGMD's prior distribution on segment allocation is less sensitive to sample size and segment length proportions, explaining its superior performance in modeling non-stationary time series.

Limitations and Future Work

  • Computational Complexity: While BGMD is more efficient than BGM, inferring complex networks can still be computationally intensive. The paper notes that complexity is polynomial in the number of nodes and time points, which is a significant advantage over exponential complexity, but practical feasibility depends on implementation efficiency and computational resources.
  • Unsupervised Learning: The authors acknowledge that for some applications, a supervised approach that uses prior biological knowledge for data segmentation before inference (as described in [27]) might be ideal. However, they chose to demonstrate the power of unsupervised learning with their model.
  • Future Work: The paper implicitly suggests that further exploration of the prior distributions and their impact on inference could be beneficial. The success of BGMD in modeling non-stationary processes also opens avenues for applying it to a wider range of dynamic biological systems.

Summary updated

Abstract

An important objective in systems biology is to infer gene regulatory networks from postgenomic data, and dynamic Bayesian networks have been widely applied as a popular tool to this end. The standard approach for nondiscretised data is restricted to a linear model and a homogeneous Markov chain. Recently, various generalisations based on changepoint processes and free allocation mixture models have been proposed. The former aim to relax the homogeneity assumption, whereas the latter are more flexible and, in principle, more adequate for modelling nonlinear processes. In our paper, we compare both paradigms and discuss theoretical shortcomings of the latter approach. We show that a model based on the changepoint process yields systematically better results than the free allocation model when inferring nonstationary gene regulatory processes from simulated gene expression time series. We further cross-compare the performance of both models on three biological systems: macrophages challenged with viral infection, circadian regulation in Arabidopsis thaliana, and morphogenesis in Drosophila melanogaster.
BibTeX
@inproceedings{grzegorcyzk2010modelling,
  author = {Marco Grzegorcyzk and Dirk Husmeier and Jörg Rahnenführer},
  title = {{Modelling Nonstationary Gene Regulatory Processes}},
  booktitle = {Advances},
  year = {2010},
  doi = {10.1155/2010/749848},
}
Figure