Data-Driven Reduction for Multiscale Stochastic Dynamical Systems

Carmeline J. Dsilva, Ronen Talmon, C. W. Gear, Ronald R. Coifman, Ioannis G. KevrekidisarXiv 2015 · DOI · arXiv

Abstract

Multiple time scale stochastic dynamical systems are ubiquitous in science and engineering, and the reduction of such systems and their models to only their slow components is often essential for scientific computation and further analysis. Rather than being available in the form of an explicit analytical model, often such systems can only be observed as a data set which exhibits dynamics on several time scales. We will focus on applying and adapting data mining and manifold learning techniques to detect the slow components in such multiscale data. Traditional data mining methods are based on metrics (and thus, geometries) which are not informed of the multiscale nature of the underlying system dynamics; such methods cannot successfully recover the slow variables. Here, we present an approach which utilizes both the local geometry and the local dynamics within the data set through a metric which is both insensitive to the fast variables and more general than simple statistical averaging. Our analysis of the approach provides conditions for successfully recovering the underlying slow variables, as well as an empirical protocol guiding the selection of the method parameters.

Introduction

In complex systems—ranging from atmospheric patterns to biochemical pathways—dynamics often occur across vastly different time scales. Fast, stochastic fluctuations (noise) often mask the underlying slow evolution (the signal) that researchers wish to analyze. To make these systems computationally tractable, model reduction is necessary. However, traditional reduction techniques like averaging or homogenization require explicit physical equations. In many modern applications, we possess high-dimensional datasets but lack the underlying governing equations. This paper presents a data-driven framework for stripping away fast noise to reveal slow, meaningful variables using manifold learning and local invariant metrics.

The Challenge of Multiscale Data

Standard data mining tools, such as Principal Component Analysis (PCA) or basic Diffusion Maps, typically rely on Euclidean distance. In multiscale systems, the fast variables often exhibit much higher variance than the slow ones. Consequently, Euclidean-based methods are "distracted" by this fast noise, incorrectly identifying it as the most significant feature and failing to recover the slow manifold. To overcome this, a metric is needed that is insensitive to fast-scale jitter while remaining sensitive to slow-scale trends.

Methodology: Local Invariant Metrics and Diffusion Maps

The proposed approach utilizes a two-step geometry-aware process:

  1. Mahalanobis Distance for Local Invariance: Instead of straight-line Euclidean distance, the method employs a locally-informed Mahalanobis distance. By normalizing distances by the local covariance (the "jitter"), the method effectively squashes the directions dominated by fast noise and stretches the directions where slow variables evolve. This covariance is estimated using "simulation bursts"—short, high-frequency snippets of data.
  2. Diffusion Maps: This manifold learning technique organizes data points based on their evolutionary likelihood. By integrating the Mahalanobis distance into the Diffusion Maps framework, the algorithm constructs a global coordinate system that captures only the slow manifold.

Parameter Selection and Empirical Protocol

A critical challenge in this data-driven approach is selecting the observation time window ($\delta t$) and the kernel scale ($\sigma_{kernel}$). As illustrated in the empirical protocol, $\delta t$ must be large enough to capture the local noise distribution but small enough that system nonlinearities do not invalidate the local linear approximation. Similarly, $\sigma_{kernel}$ must be tuned to respect the curvature of the underlying manifold.

The empirical protocol for setting the observation window and kernel scale ensures that local approximations remain accurate despite system nonlinearities and manifold curvature.

Results: Recovering the Slow Manifold

The method was validated using synthetic Stochastic Differential Equation (SDE) models, including linear multiscale systems and nonlinear "half-moon" coupled systems. In cases where Euclidean metrics failed by focusing on the high-variance fast noise, the Mahalanobis-based approach successfully "collapsed" the fast variables to reveal the underlying slow dynamics.

Comparison between raw multiscale data (b) and the recovered slow manifold (a). The geometric approach effectively collapses the fast-moving noise into a single, coherent slow trajectory.

One of the most significant findings is the method's invariance to the observation function. Because the Mahalanobis distance "undoes" local warping, the approach can successfully fuse data from different types of sensors, even when the underlying variables are transformed by complex, nonlinear measurement functions.

Conclusion

This data-driven reduction framework provides a robust alternative to equation-based homogenization. By treating noise as a feature of local geometry rather than a nuisance to be averaged out, the method allows for the discovery of slow variables in high-dimensional, multiscale datasets. Future work may extend this to "equation-free" modeling, where discovered variables are used to build accelerated simulators for complex systems that are currently beyond our modeling reach.

Summary updated

Abstract

Multiple time scale stochastic dynamical systems are ubiquitous in science and engineering, and the reduction of such systems and their models to only their slow components is often essential for scientific computation and further analysis. Rather than being available in the form of an explicit analytical model, often such systems can only be observed as a data set which exhibits dynamics on several time scales. We will focus on applying and adapting data mining and manifold learning techniques to detect the slow components in such multiscale data. Traditional data mining methods are based on metrics (and thus, geometries) which are not informed of the multiscale nature of the underlying system dynamics; such methods cannot successfully recover the slow variables. Here, we present an approach which utilizes both the local geometry and the local dynamics within the data set through a metric which is both insensitive to the fast variables and more general than simple statistical averaging. Our analysis of the approach provides conditions for successfully recovering the underlying slow variables, as well as an empirical protocol guiding the selection of the method parameters.
BibTeX
@misc{dsilva2015data,
  author = {Carmeline J. Dsilva and Ronen Talmon and C. W. Gear and Ronald R. Coifman and Ioannis G. Kevrekidis},
  title = {{Data-Driven Reduction for Multiscale Stochastic Dynamical Systems}},
  year = {2015},
  eprint = {1501.05195},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1501.05195},
}

Loading related content...

Figure