Predicting multiple observations in complex systems through low-dimensional embeddings

Tao Wu, Xiangyun Gao, Feng An, Xiaotian Sun, Haizhong An, Zhen Su, Shraddha Gupta, Jianxi Gao, Jürgen KurthsNature Communications 2024 · DOI

TL;DR

This paper introduces FRMM, a novel data-driven framework that predicts all components of complex, high-dimensional systems by mapping their low-dimensional representations. By combining feature embedding and delay embedding, FRMM creates a generalized predictor that overcomes the "curse of dimensionality" and shows promising results across diverse real-world applications.

What Problem Does This Paper Address?

Predicting the future states of complex dynamical systems is a significant challenge, especially when dealing with systems that have many interconnected components (high dimensionality). Existing methods often struggle with:

  • Predictor Uncertainty: Selecting the right predictors for a target variable can be difficult. Using all variables can introduce noise and redundancy, while empirical selection might miss crucial information.
  • Model Uncertainty: Many approaches require training separate models for each component, leading to high computational costs and a lack of a unified framework.
  • Forecasting Multiple Observations: Most methods are validated on predicting only a single or a few variables, not the entire system's behavior.

The core problem is the lack of a generalized, reliable framework to forecast all components of a complex system simultaneously, particularly when faced with high dimensionality and limited knowledge of the underlying physics.

Methods

The paper proposes a novel framework called Feature-and-Reconstructed Manifold Mapping (FRMM). It's a data-driven and model-free approach that leverages two key techniques:

  1. Delay Embedding: Based on Takens' embedding theorem, this technique reconstructs a low-dimensional, topologically isomorphic manifold (the "reconstructed manifold") of the original high-dimensional system from a single time series. This is done by creating state vectors from time-delayed observations of that series.
  2. Feature Embedding: This uses manifold learning algorithms (like diffusion maps, ISOMAP, LLE, Laplacian, LTSA) to find a low-dimensional representation (the "feature manifold") that preserves the geometric features (e.g., nearest neighbors) of the original system.

The core idea is that both delay embedding and feature embedding can reveal low-dimensional structures that are isomorphic to the original system. FRMM then establishes a mapping between these two low-dimensional representations. The feature manifold is then used as a generalized predictor to forecast the future states of all components of the original system. Gaussian process regression is employed to train the mapping between these manifolds.

Datasets

The FRMM framework was evaluated on a variety of datasets:

  • Model Systems:
    • 3-dimensional Lorenz system
    • 90-dimensional coupled Lorenz system (both time-invariant and time-varying dynamics)
  • Real-World Systems:
    • Indian monsoon (daily wind speed data at 850 hPa)
    • Electroencephalogram (EEG) signals (64-dimensional subsystem from a healthy participant)
    • Foreign exchange market (daily closing prices of 70 currencies against the US dollar)
    • Traffic speed data (collected from 207 loop detectors in Los Angeles County)
    • 46 stock indices from global stock markets

Key Insights and Ideas

  • Unified Predictor: The most significant insight is the development of a generalized predictor (the low-dimensional feature manifold) that can be used to forecast all components of a complex system, rather than needing individual predictors for each.
  • Isomorphism is Key: The framework relies on the theoretical grounding that low-dimensional embeddings (from both delay and feature embedding) can capture the essential topological structure of high-dimensional systems, and importantly, these embeddings are isomorphic to each other.
  • Overcoming Dimensionality: FRMM effectively bypasses the "curse of dimensionality" by working in a reduced low-dimensional space, making predictions computationally feasible and more robust.
  • Data-Driven Power: The model-free nature of FRMM means it can be applied to systems where the underlying physics are unknown or too complex to model explicitly, relying solely on observed time series data.

Key Results

FRMM demonstrated strong performance across various benchmarks:

  • Model Systems:
    • Achieved high prediction accuracies (average $\rho$ of 0.91) for the 3D Lorenz system.
  • Showed reliable predictions for the 90D Lorenz system, even with time-varying dynamics (average $\rho$ of 0.73).
  • Real-World Systems:
    • Indian Monsoon: Average $\rho$ of 0.86 for 20-day ahead predictions.
  • EEG Signals: Average $\rho$ of 0.92 for 20-second ahead predictions.
  • Foreign Exchange Market: Average $\rho$ of 0.89 for 20-day ahead predictions.
  • Traffic Speed: Achieved 10-step ahead predictions with an average $\rho$ of 0.78 for 86% of components.
  • Stock Indices: Showed reliable predictions for 46 global stock indices.

The framework also proved robust to variations in training sample size, noise levels, and input data length.

Limitations and Future Work

  • Synchronous Prediction: FRMM, like other STI-based frameworks, cannot predict all components synchronously. It requires training a suitable mapping for each target variable.
  • Critical Transitions: The framework struggles to predict abrupt, rapid, or irreversible transitions (tipping points) in systems. The authors suggest that poor performance might indicate an impending critical transition, which could be a direction for future research.
  • Feature Embedding Choice: While FRMM works with various feature embedding algorithms, there isn't a single "golden rule" for selecting the optimal one for all systems, as real-world dynamics are diverse.
  • Long-Term Prediction: The paper notes limitations in making very long-term predictions, as the number of predictable steps is bounded by $(E-1)\tau$.

Future work could focus on developing methods to identify tipping points and improve synchronous prediction capabilities. The authors also suggest that neural network algorithms could potentially be used for training the mappings, though they highlight the trade-offs in terms of cost and interpretability compared to Gaussian process regression.

Summary updated

Abstract

Forecasting all components in complex systems is an open and challenging task, possibly due to high dimensionality and undesirable predictors. We bridge this gap by proposing a data-driven and model-free framework, namely, feature-and-reconstructed manifold mapping (FRMM), which is a combination of feature embedding and delay embedding. For a high-dimensional dynamical system, FRMM finds its topologically equivalent manifolds with low dimensions from feature embedding and delay embedding and then sets the low-dimensional feature manifold as a generalized predictor to achieve predictions of all components. The substantial potential of FRMM is shown for both representative models and real-world data involving Indian monsoon, electroencephalogram (EEG) signals, foreign exchange market, and traffic speed in Los Angeles Country. FRMM overcomes the curse of dimensionality and finds a generalized predictor, and thus has potential for applications in many other real-world systems.
BibTeX
@inproceedings{wu2024predicting,
  author = {Tao Wu and Xiangyun Gao and Feng An and Xiaotian Sun and Haizhong An and Zhen Su and Shraddha Gupta and Jianxi Gao and Jürgen Kurths},
  title = {{Predicting multiple observations in complex systems through low-dimensional embeddings}},
  booktitle = {Nature Communications},
  year = {2024},
  doi = {10.1038/s41467-024-46598-w},
}
Figure