Efficient Bayesian Inference for Discretely Observed Continuous Time Markov Chains

Tao Tang, Lachlan Astfalck, David B. DunsonarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces a novel Bayesian framework for efficiently inferring the dynamics of continuous-time Markov chains (CTMCs) when only discrete observations are available. By employing a "pseudo-likelihood" that cleverly bypasses the intractable exact likelihood, their method scales well to medium-high dimensional state spaces and offers theoretical guarantees for its accuracy.

What Problem Does This Paper Address?

Continuous-time Markov chains (CTMCs) are powerful tools for modeling systems that evolve over time, such as biological processes, financial markets, or physical phenomena. However, inferring the underlying dynamics (represented by the generator matrix $L$) from data is notoriously difficult when we only observe the system's state at discrete time points. The core challenge is that the exact likelihood function, which connects the observed data to the model parameters, is intractable. This intractability arises because the exact transitions between states are unknown, and existing methods often become computationally prohibitive as the number of states or observations grows. Furthermore, even if we could compute the likelihood, inferring $L$ from discrete observations can lead to non-identifiable solutions, meaning different $L$ matrices could explain the same data, making extrapolation and interpretation problematic.

Methods

The authors propose a scalable Bayesian approach that tackles these challenges by introducing a pseudo-likelihood. Instead of directly modeling the intractable generator matrix $L$, they model the transition probability matrix $P$ (which relates states at discrete time steps) and $L$ jointly. A key innovation is the use of a biorthogonal spectral decomposition of $L$. This decomposition provides a structured way to represent $L$ and ensures that the inferred $L$ is "embeddable" (i.e., it can generate a valid continuous-time process).

The pseudo-likelihood combines two terms:

  1. A standard log-likelihood based on the observed transitions in the data, which depends on $P$.
  2. A regularization term that encourages agreement between $P$ and its spectral representation derived from $L$.

This approach allows for an efficient Gibbs sampling procedure to estimate the model parameters. The method also incorporates theoretical justifications, including a Bernstein-von Mises theorem for the transition probability matrix and posterior consistency for the spectral parameters of the generator.

Datasets

Not discussed in the paper. The paper focuses on synthetic data generated from known CTMCs and a simulated metastable diffusion process.

Key Insights and Ideas

  • Bypassing Intractability with Pseudo-Likelihood: The core idea of using a pseudo-likelihood that links the transition matrix $P$ and the generator $L$ via a spectral decomposition is a clever way to circumvent the computational burden of the exact likelihood.
  • Biorthogonal Spectral Decomposition for Identifiability: By enforcing a biorthogonal spectral decomposition of $L$, the authors ensure that the inferred generator is structurally valid and identifiable, addressing a fundamental problem in CTMC inference.
  • Scalability to Medium-High Dimensions: The proposed method is designed to scale well with the number of states, making it applicable to problems previously out of reach for exact Bayesian methods.
  • Theoretical Guarantees: Establishing a Bernstein-von Mises theorem and posterior consistency provides strong theoretical backing for the proposed pseudo-likelihood approach, assuring users of its statistical validity.

Key Results

  • Computational Efficiency: The method is significantly faster than existing approaches (like BLANDT and MH-RIVA) for simulating CTMCs, especially as the number of observations ($n$) increases. For example, in simulations, their method (BIGMAC) shows run-times that are nearly invariant to $n$, while others scale poorly.
  • Accuracy: The model achieves comparable accuracy to existing methods in estimating the generator matrix $L$, as measured by the average Frobenius norm between posterior means and the true $L$. For instance, in Table 2, BIGMAC's performance is consistently close to BLANDT (which is treated as a proxy for ground truth).
  • Effective Sample Size (ESS): While BIGMAC and BLANDT show similar mixing properties, MH-RIVA suffers from markedly lower ESS, indicating poorer sampling efficiency.
  • Application to Metastable Diffusion: The method successfully models a challenging metastable diffusion process, accurately recovering the dynamics of the system and demonstrating its applicability to complex, higher-dimensional problems.

Limitations and Future Work

  • Assumption of Regular Observation Times: The current framework assumes that observations are made at regular time intervals ($\Delta$). The authors suggest future work could explore relaxing this assumption.
  • Assumption of Biorthogonal Generator: The method relies on the assumption that the generator $L$ has a biorthogonal spectral decomposition. Future research could investigate relaxing this assumption.
  • Further Exploration of Metastable Systems: The paper demonstrates the method's utility on a metastable diffusion process and suggests this is a promising area for future applications.

Summary updated

Abstract

Inference for continuous-time Markov chains (CTMCs) becomes challenging when the process is only observed at discrete time points. The exact likelihood is intractable, and existing methods often struggle even in medium-dimensional state-spaces. We propose a scalable Bayesian framework for CTMC inference based on a pseudo-likelihood that bypasses the need for the full intractable likelihood. Our approach jointly estimates the probability transition matrix and a biorthogonal spectral decomposition of the generator, enabling an efficient Gibbs sampling procedure that obeys embeddability. Existing methods typically integrate out the unobserved transitions, which becomes computationally burdensome as the number of data or dimensions increase. The computational cost of our method is near-invariant in the number of data and scales well to medium-high dimensions. We justify our pseudo-likelihood approach by establishing theoretical guarantees, including a Bernstein-von Mises theorem for the probability transition matrix and posterior consistency for the spectral parameters of the generator. Through simulation and applications, we showcase the flexibility and robustness of our approach, offering a tractable and scalable approach to Bayesian inference for CTMCs.
BibTeX
@misc{tang2025efficient,
  author = {Tao Tang and Lachlan Astfalck and David B. Dunson},
  title = {{Efficient Bayesian Inference for Discretely Observed Continuous Time Markov Chains}},
  year = {2025},
  eprint = {2507.16756},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.16756},
}

Loading related content...

Figure