Deep reconstruction of strange attractors from time series

William GilpinNeurIPS 2020

Introduction

In many experimental fields—from cardiology to meteorology—scientists are often forced to observe complex, multi-dimensional systems through a single "keyhole." We might only be able to measure a single variable, like a heart rate monitor's output or a single temperature sensor, even though the underlying system is governed by many hidden, interacting variables.

Historically, researchers have used "state space reconstruction" (based on Takens’ Theorem) to infer these hidden variables by analyzing time-lagged versions of the available data. However, these classical methods are notoriously sensitive to noise and require manual, often tedious, tuning of hyperparameters. Furthermore, while most modern machine learning focuses on dimensionality reduction (compressing complex data into a simple representation), this challenge requires solving the inverse problem: taking low-dimensional observations and reconstructing the higher-dimensional "engine" driving them.

This paper introduces a deep learning method to reconstruct the full, high-dimensional "strange attractor" of a chaotic system using only a single stream of measurement data. By training an autoencoder with a novel "false-nearest-neighbor" loss function, the researchers successfully "unfold" complex dynamics from simple time series, outperforming classical techniques in both accuracy and noise robustness.

The Deep Reconstruction Framework

The authors propose a "Deep Reconstruction" architecture that bridges classical dynamic systems theory with modern deep learning.

Architecture of the Deep Reconstruction framework

The methodology consists of three main components:

  1. Input Representation: The univariate time series is first converted into a Hankel matrix. This process essentially stacks time-lagged versions of the signal to provide the network with a sliding window of the system's history.
  2. Architecture: A stacked autoencoder (utilizing either LSTM or MLP layers) processes the matrix. The encoder maps the input to a latent space, and the decoder attempts to reconstruct the original signal.
  3. The Innovation ($L_{FNN}$): The core contribution of the paper is a novel False-Nearest-Neighbor (FNN) loss function.

To understand $L_{FNN}$, imagine a 3D knot's shadow cast onto a 2D wall. In the 2D projection, parts of the string appear to touch or cross (they look like neighbors), but in the actual 3D space, they are far apart. These are "false neighbors." The $L_{FNN}$ loss acts as a regularizer that penalizes the network if points that are close in the latent space "fly apart" when the dimensionality is slightly increased. This continuous, differentiable heuristic forces the network to find a latent space just large enough to "unfold" the attractor without adding redundant, noisy dimensions.

Key Results and Benchmarks

The method was validated across a diverse range of systems, including synthetic chaotic systems (Lorenz, Rössler, Lotka-Volterra, Torus), experimental physics data (a chaotic double pendulum), and real-world datasets like human electrocardiograms (ECG), Old Faithful geyser eruptions, and neural spiking rates.

Benchmarking results across various chaotic systems
  • Superior Benchmarking: The proposed method—specifically the LSTM-FNN variant—consistently outperformed baselines like Eigen-Time-Delay (ETD) and Time-lagged Independent Component Analysis (tICA). It achieved higher scores across multiple metrics, including forecasting accuracy, topological similarity, and fractal dimension.
  • Learned Filtering and Robustness: Unlike classical methods that struggle with noise, the autoencoder acts as a learned filter. It identifies the salient signal and ignores stochastic fluctuations. In simulations with high noise (Brownian motion forcing), the regularized model maintained high predictive accuracy at long time horizons where other methods failed.
  • Discovery in Real Data: The model successfully extracted meaningful geometries from real-world "unknown" systems. In ECGs, it reconstructed a nested loop geometry consistent with analytical heart models. In neural spiking data, it identified a "double-limit-cycle" structure, and in Old Faithful data, it revealed a low-dimensional quasi-periodic attractor that supports long-standing hypotheses about the geyser's internal pressure dynamics.

Limitations and Future Work

While highly effective, the approach has some limitations. Training a neural network is significantly more computationally expensive than classical linear methods, which might be a drawback for very simple or small datasets. Additionally, while the method reduces the overall number of hyperparameters, the strength of the regularizer ($\lambda$) still requires tuning.

Looking forward, the authors suggest that these reconstructed "latent coordinates" could eventually be used to learn explicit differential equations ($dy/dt = f(y)$) directly from data. This presents an exciting pathway to bridge the gap between black-box deep learning and interpretable classical physics.

Summary updated

BibTeX
@inproceedings{gilpin2020deep,
  author = {William Gilpin},
  title = {{Deep reconstruction of strange attractors from time series}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2020},
}
Figure