Bayesian Inference for Change Points in Dynamical Systems with Reusable States - a Chinese Restaurant Process Approach

Florian Stimberg, Andreas Ruttor, Manfred OpperAISTATS 2012

Introduction

Many complex systems, ranging from financial markets to gene regulatory networks, exhibit non-stationary dynamics where behavior shifts suddenly between distinct regimes. Conventional change-point models often assume that each transition leads to an entirely new state with unique parameters. However, many physical and biological systems are reentrant; for instance, a gene may toggle between "active" and "inactive" modes. Treating every temporal segment as unique fails to leverage the information available when a system re-enters a previously observed state. This paper addresses the challenge of performing inference in systems where the number of states is unknown and parameters can be reused over time.

Methodology

The proposed framework models system dynamics using a linear stochastic differential equation (SDE), specifically a set of independent Ornstein-Uhlenbeck (OU) processes. The core innovation is a hierarchical Bayesian prior that governs the evolution of system parameters:

  1. Change Points: The times at which the system jumps between states are modeled as a Poisson process.
  2. Reusable States via the Chinese Restaurant Process (CRP): To allow for state reuse, the authors utilize a CRP prior. In this formulation, each new time segment (a "customer") can either join an existing "table" (reusing a previously seen parameter set) or start a new one (creating a new state). This effectively implements a Dirichlet Process prior, allowing the model to adaptively determine the number of states.
  3. Efficient Inference: A Metropolis-within-Gibbs MCMC sampler is developed for posterior estimation. To improve efficiency, the authors employ "Rao-Blackwellization," analytically integrating out the latent system states so the sampler focuses on change points and CRP assignments.
Graphical model of the proposed Bayesian framework, showing the hierarchical relationship between the CRP prior (pi), change points (tau), and the dynamical system parameters (A).

Experimental Evaluation

The model was evaluated on two primary datasets:

  • Synthetic Data: A two-dimensional OU process featuring 9 jumps across 5 distinct states, designed to test the model's ability to recover complex hidden structures.
  • Yeast Metabolic Cycle: Real-world microarray gene expression data consisting of 10 time series. This was used to infer underlying transcriptional regulation states and validate the model's biological relevance.

Results and Discussion

The CRP-based approach yielded several significant improvements over standard models. By recognizing when segments belong to the same state, the model can pool data across disparate time periods, significantly enhancing the precision of parameter estimates—particularly for short segments that lack sufficient data in isolation.

On synthetic data, the model reduced parameter estimation error by over 15% compared to non-reusable change-point models. It also demonstrated superior robustness to observation noise, showing a 5% improvement by "anchoring" noisy segments to cleaner, previously observed states.

Inferred parameter trajectories A1(t) and A2(t) for the synthetic OU process. The blue lines and shaded regions represent the posterior mean and uncertainty, while the red dashed lines indicate the ground truth.

In the biological application to yeast data, the model identified three distinct states involved in transcriptional regulation. These findings align with established biological knowledge regarding the combinatorial logic of transcription factors FHL1 and RAP1 (e.g., off/off, on/off, and on/on configurations).

Despite these successes, computational scaling remains a hurdle. While a C++ implementation achieved a 12x speedup over Matlab (processing 1 million samples in approximately 40 minutes), extremely high-dimensional systems require further optimization. Future work may extend this framework to other stochastic processes, such as the Cox-Ingersoll-Ross model, or allow for partial state changes where only a subset of parameters jump at any given time.

Conclusion

By integrating a Chinese Restaurant Process prior into a change-point detection framework, this research provides a powerful tool for analyzing reentrant dynamical systems. The ability to automatically discover the number of states and pool information across time segments leads to more robust and biologically meaningful insights into complex temporal data.

Summary updated

BibTeX
@article{stimberg2012bayesian,
  author = {Florian Stimberg and Andreas Ruttor and Manfred Opper},
  title = {{Bayesian Inference for Change Points in Dynamical Systems with Reusable States - a Chinese Restaurant Process Approach}},
  journal = {AISTATS},
  year = {2012},
}
Figure