Optimal Data-Driven Estimation of Generalized Markov State Models for Non-Equilibrium Dynamics

Péter Koltai, Hao Wu, Frank Noé, Christof SchütteComput 2018 · OpenReview

TL;DR

This paper introduces a unified framework for building simplified models (generalized Markov State Models or GMSMs) of complex systems, even when they are not in a stable equilibrium state. By focusing on minimizing "propagation error," the authors provide a robust method to extract essential dynamics from simulation data, applicable to both equilibrium and non-equilibrium scenarios.

What Problem Does This Paper Address?

Understanding the long-term behavior of complex systems, especially in fields like molecular dynamics, is crucial. Traditionally, Markov State Models (MSMs) have been powerful tools for this, but they primarily work for systems in statistical equilibrium. Many real-world systems, however, are not in equilibrium. They might be subjected to external forces that change over time, be in a transient phase, or simply not exhibit a clear equilibrium distribution. Existing methods struggle to accurately capture the dynamics of these non-equilibrium systems, leading to a need for more general and robust modeling approaches. This paper aims to bridge this gap by extending the power of MSMs to these more complex, non-equilibrium situations.

Methods

The core of this paper lies in generalizing the concept of Markov State Models (MSMs) to "generalized Markov State Models" (GMSMs). The authors achieve this by framing the problem as finding an optimal low-rank approximation of "transfer operators." Transfer operators are mathematical tools that describe how probability distributions evolve over time in a dynamical system.

For equilibrium systems, the approach leverages the concept of "metastable sets" and approximates the transfer operator by projecting it onto the dominant eigenfunctions. This results in a GMSM that captures the slowest relaxation timescales.

For non-equilibrium systems, the paper introduces the concept of "minimal propagation error" as the key criterion for building a GMSM. Instead of relying on equilibrium properties like detailed balance, the GMSM is constructed to minimize how much errors in the model propagate over time. This is achieved by finding optimal projections onto subspaces spanned by the dominant singular vectors of the transfer operator. The paper also introduces "coherent sets" as a generalization of metastable sets for non-equilibrium dynamics, providing a physical interpretation for the GMSM.

The data-driven estimation of these GMSMs is achieved through a time-lagged canonical correlation analysis (TCCA) algorithm, which leverages Monte Carlo estimates from trajectory data.

Datasets

The paper uses numerical examples to illustrate its methods. These include:

  • A double-well potential system (used for equilibrium dynamics).
  • A shifting triple-well potential system (used for time-inhomogeneous dynamics).
  • A non-reversible system with a circular driving force (used for non-reversible, non-stationary dynamics).

No new benchmark datasets are introduced; rather, these are standard or well-defined model systems used for evaluation.

Key Insights and Ideas

  • Unified Framework: The paper elegantly unifies the modeling of both equilibrium and non-equilibrium dynamics under the umbrella of optimal low-rank approximation of transfer operators. This provides a consistent theoretical foundation.
  • Propagation Error as a Guiding Principle: Shifting the focus from reproducing exact timescales to minimizing "propagation error" offers a more robust and general approach for non-equilibrium systems, where traditional notions of timescales can be ill-defined.
  • Coherent Sets as a Generalization of Metastability: The introduction of "coherent sets" provides a physically intuitive way to interpret the GMSMs for non-equilibrium systems, analogous to how "metastable sets" are used for equilibrium systems.
  • Data-Driven Robustness: The TCCA algorithm demonstrates how to effectively build these GMSMs from finite simulation data, even when the data doesn't perfectly represent the system's true stationary distribution.

Key Results

  • The paper presents a method to construct generalized Markov State Models (GMSMs) that accurately approximate the dominant dynamics of systems, even when they are far from equilibrium.
  • For equilibrium systems, the GMSMs reproduce the dominant time scales, as demonstrated with a double-well potential example, yielding a rank-2 MSM with an approximate time scale of 82.4.
  • For time-inhomogeneous systems, the method successfully identifies coherent sets and constructs GMSMs that capture the essential transitions between these sets, as shown with a shifting triple-well potential example, resulting in a rank-3 GMSM.
  • For non-reversible systems with non-stationary data, the approach constructs a GMSM by minimizing propagation error, leading to a rank-7 GMSM that captures the dominant dynamics and provides insight into the system's invariant distribution.

Limitations and Future Work

  • Interpretation of Eigenvalues/Singular Values: While the paper focuses on minimizing propagation error, the direct physical interpretation of eigenvalues and singular values in non-equilibrium settings can be more complex than in equilibrium cases. The paper acknowledges that the "metastable" interpretation might be ambiguous for cyclic systems.
  • Computational Cost: Estimating GMSMs from large datasets can be computationally intensive, especially when dealing with high-dimensional systems or requiring a large number of basis functions.
  • Future Work: The authors suggest that their framework can be extended to other types of non-equilibrium systems and that further exploration of the physical interpretation of coherent sets and their relation to metastability is warranted. They also hint at applications in areas like molecular dynamics where these models are highly relevant.

Summary updated

Abstract

There are multiple ways in which a stochastic system can be out of statistical equilibrium. It might be subject to time-varying forcing; or be in a transient phase on its way towards equilibrium; it might even be in equilibrium without us noticing it, due to insufficient observations; and it even might be a system failing to admit an equilibrium distribution at all. We review some of the approaches that model the effective statistical behavior of equilibrium and non-equilibrium dynamical systems, and show that both cases can be considered under the unified framework of optimal low-rank approximation of so-called transfer operators. Particular attention is given to the connection between these methods, Markov state models, and the concept of metastability, further to the estimation of such reduced order models from finite simulation data. All these topics bear an important role in, e.g., molecular dynamics, where Markov state models are often and successfully utilized, and which is the main motivating application in this paper. We illustrate our considerations by numerical examples.
BibTeX
@inproceedings{koltai2018optimal,
  author = {Péter Koltai and Hao Wu and Frank Noé and Christof Schütte},
  title = {{Optimal Data-Driven Estimation of Generalized Markov State Models for Non-Equilibrium Dynamics}},
  booktitle = {Comput},
  year = {2018},
  url = {https://openreview.net/forum?id=aXByFaWx5x},
}

Loading related content...

Figure