Predicting the future of a nonlinear system—whether it is the turbulent churning of fluid flow, the fluctuating load on a power grid, or the behavior of a chemical reactor—is notoriously difficult. In a linear system, small changes yield proportional, predictable results. In a nonlinear system, however, tiny variations can trigger dramatic, chaotic transformations. This sensitivity is particularly evident in fluid mechanical instabilities, as discussed by Shruti Mishra et al., where minor initial perturbations completely reshape the system's trajectory.
Because errors compound exponentially over long time horizons, standard regression models often fail to maintain accuracy when forecasting nonlinear systems (Shaowu Pan et al.). This creates a fundamental challenge: how can we model complex, real-world dynamics without succumbing to the compounding errors of nonlinear approximations?
The core insight of this research direction is a change of perspective. Instead of trying to fit complex, curved trajectories in the system's original state space, we can map the data into a new coordinate system where the trajectories evolve along straight lines. By finding a linear embedding for nonlinear dynamics, we can leverage the entire toolkit of linear algebra—such as matrix exponentials and spectral analysis—to model and forecast otherwise unpredictable systems.
The Koopman Operator: Trading Nonlinearity for Dimensions
The mathematical foundation for this idea traces back to Koopman operator theory. Suppose we have a system whose state $x_t$ updates over time via a complex, nonlinear function:
$$x_{t+1} = f(x_t)$$
Instead of tracking the state $x_t$ directly, what if we observe the system through a set of measurement functions, $\psi(x)$, known as observables? The Koopman operator, $\mathcal{K}$, is an infinite-dimensional linear operator that acts on these observables to advance them forward in time:
$$\psi(x_{t+1}) = \mathcal{K}\psi(x_t)$$
The profound trick here is a trade-off: we exchange a finite-dimensional, nonlinear problem for an infinite-dimensional, but completely linear one.
Nonlinear Space (Hard): x_t ------------[ f(x) ]------------> x_{t+1}
| ^
[ Lift / Embed ] | [ Un-embed ]
v |
Linear Space (Easy): ψ(x_t) -----------[ Matrix K ]----------> ψ(x_{t+1})
To make this mathematically rigorous and practically useful for data-driven modeling, Milan Korda et al. provided convergence proofs for Extended Dynamic Mode Decomposition (EDMD). They demonstrated that as you increase both the number of data samples and the dictionary of observable functions, the data-driven matrix approximations converge directly to the true, underlying Koopman operator, enabling accurate predictions over finite time horizons.
Discovering the Coordinates with Deep Learning
While theory guarantees that an infinite-dimensional linear operator exists, computers require finite representations. The central engineering challenge is finding a compact, low-dimensional set of observables that form an invariant subspace—a coordinate system where the dynamics remain linear without needing infinite dimensions.
Rather than manually guessing these observable functions, modern frameworks use deep neural networks as encoders to discover them automatically from data. Samuel R. Moore et al. demonstrated that this process can be automated to extract clean, low-dimensional linear embeddings directly from messy, experimental real-world observations.
However, unconstrained neural networks can introduce new failure modes, such as learning representations that cannot be mapped back to the original physical space, or models that drift into instability. Researchers have addressed these challenges through several key architectural innovations:
- Invertibility: To ensure that predictions made in the linear embedding space can be translated back to physical states, Yuhuang Meng et al. introduced FlowDMD. By utilizing invertible neural networks (coupling flows), their framework guarantees a bi-directional mapping between the physical state and the linear embedding, improving reconstruction accuracy.
- Guaranteed Stability: Linear models can still explode if their eigenvalues are unconstrained. To prevent this, Shaowu Pan et al. proposed a physics-informed probabilistic framework that embeds stability guarantees directly into the learned linear representations.
- Regularization: To mitigate error propagation over extended timelines, Shaowu Pan et al. introduced a Jacobian regularization technique for feedforward networks, penalizing the Frobenius norm of the network's Jacobian to stabilize long-term rollouts.
Scaling and Structuring the Embeddings
For massive, high-dimensional systems like power grids or climate models, learning a global linear embedding can become computationally prohibitive. Recent work focuses on structuring these operators to make them highly efficient.
One approach is parallelization. Masih Haseli et al. developed a distributed framework that learns Koopman eigenfunctions and invariant subspaces in parallel across a network of processors, achieving massive computational speedups while maintaining rigorous theoretical guarantees.
Another strategy replaces heavy, attention-based sequence models with structured recurrent architectures. Yitian Zhang et al. bridged Koopman operator theory with modern linear Recurrent Neural Networks (RNNs) to create SKOLR. By splitting complex, chaotic signals into distinct frequency bands and processing them with parallel linear RNNs, they achieved state-of-the-art forecasting accuracy using only a fraction of the memory and compute required by Transformers.
Managing Real-World Complexity: Noise, Delays, and Uncertainty
When moving from clean simulations to real-world data, linear embedding frameworks must contend with noise, unobserved variables, and structural shifts.
Stochasticity and Hidden States
Real systems are rarely perfectly deterministic. To model noisy, stochastic dynamics, David Wingate et al. developed the Kernel Predictive Linear Gaussian (KPLG) model, which applies kernel methods to lift noisy data into a space where linear Gaussian filtering can track unobserved states. For industrial anomaly detection, Cheng Feng et al. combined neural system identification with an Unscented Kalman Filter (NSIBF), using deep networks to isolate the underlying non-linear dynamics while leveraging classical state estimation to track uncertainty and flag deviations in real-time sensor data.
Variational and Sequential Inference
When the underlying system behaves as a Markov process, Frank Noe et al. established the Variational Approach for Markov Processes (VAMP). VAMP provides a rigorous framework to learn optimal, low-dimensional Markovian representations from time-series data, accommodating both reversible and irreversible processes. When uncertainties are highly non-Gaussian, Konstantinos Tatsis et al. demonstrated how Sequential Monte Carlo and particle filtering methods can be deployed to track states and parameters dynamically.
Geometric and Network Constraints
Sometimes, the continuous trajectory of a system is less important than its structural properties or discrete transitions:
- Geometric Reduction: Jason J. Bramburger combined sparse regression (SINDy) with Poincaré sections, reducing continuous, multiscale physical trajectories into discrete-time maps that isolate slow-scale dynamics.
- Time Delays: In engineering systems, time delays can interact with nonlinearities to cause instability. Harald Pfifer et al. addressed this by using Integral Quadratic Constraints (IQCs) to analyze stability when parameter-varying systems suffer from delayed feedback.
- Interconnected Networks: For large-scale networks of interacting components, such as biological oscillators, Rodolphe Sepulchre utilized dissipativity theory to establish global synchronization conditions, balancing energy restoration and dissipation across high-dimensional networks.
- Switching Interactions: To capture dependencies that change over time, Zoran Dzunic developed a Bayesian switching interaction framework, allowing models to adapt as the underlying network topology of a complex system shifts.
By shifting the modeling paradigm from "fitting nonlinear curves" to "discovering linearizing coordinates," these diverse approaches collectively turn the curse of dimensionality into an advantage—enabling the scalable prediction, tracking, and control of highly complex systems.
Key Foundational Work
Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems (2006) — contributed 1 concept elements to this direction.
Tree-structured nonlinear signal modeling and prediction (1999) — contributed 1 concept elements to this direction.