Linearizing Nonlinear Dynamics in Latent Spaces

1,453 concepts732 papers4 proposals

The real world is stubbornly nonlinear. From the turbulent swirling of fluid flows to the chaotic firing of neurons, predicting how a complex system evolves over time usually requires solving intractable differential equations or training black-box neural networks that offer little structural insight.

In contrast, linear systems are a mathematician's dream. We can solve them exactly using matrix exponentials, analyze their stability by looking at eigenvalues, and control them optimally using time-tested tools like the Linear Quadratic Regulator (LQR).

What if we could have the best of both worlds? What if we could "straighten out" a complex, nonlinear trajectory so that it behaves completely linearly? This is the core promise of Koopman operator theory combined with modern representation learning.

Lifting to a Linear Space

In 1931, Bernard Koopman proved a remarkable mathematical fact: any nonlinear dynamical system can be mapped into an infinite-dimensional space of "observable functions" where its time evolution becomes perfectly linear. Instead of tracking the physical state $x_t$ directly, we track a set of functions of the state, $\psi(x_t)$. In this infinite-dimensional space, the transition from one time step to the next is governed by a linear operator $\mathcal{K}$, such that:

$$\psi(x_{t+1}) = \mathcal{K}\psi(x_t)$$

While an infinite-dimensional operator is computationally impossible to represent exactly, modern deep learning provides a practical workaround: we can use neural networks to discover a finite-dimensional latent space that acts as an invariant subspace for the Koopman operator.

As demonstrated in Deep learning for universal linear embeddings of nonlinear dynamics by Bethany Lusch et al., this can be achieved using a deep autoencoder architecture. An encoder network maps the high-dimensional nonlinear state $x$ into a low-dimensional latent coordinate $z = \psi(x)$. Within this latent space, the dynamics are explicitly constrained to evolve via simple matrix multiplication:

$$z_{t+1} = A z_t$$

Finally, a decoder network $\phi(z)$ maps the predicted latent state back to the original physical space, reconstructing $\hat{x}_{t+1}$. By training the encoder, decoder, and the transition matrix $A$ simultaneously, the network learns to "straighten out" chaotic physical trajectories into predictable, linear paths.

Why It Works: The Power of Spectral Properties

Why go through the trouble of mapping a system into a linear latent space? The answer lies in the power of linear algebra. Once the dynamics are governed by a matrix $A$, we can compute its eigenvalues and eigenvectors.

In a linear latent space, every complex trajectory is broken down into a combination of fundamental "modes"—periodic oscillations, exponential growths, or decaying curves. In Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs, Ilan Naiman et al. point out that this spectral decomposition dramatically improves the interpretability of generative models, allowing researchers to analyze system stability and growth directly by examining the spectral properties of the learned transition operator.

Furthermore, predicting the future state of the system no longer requires iteratively evaluating a complex, error-prone nonlinear function. Instead, forecasting $N$ steps into the future simplifies to computing $A^N z_0$, reducing cumulative errors and making long-term prediction mathematically trivial.

This property is incredibly powerful for tracking chaotic motion patterns. For example, in DenseKoopman: A Plug-and-Play Framework for Dense Pedestrian Trajectory Prediction, Han Jiang et al. show that transforming nonlinear pedestrian trajectories into a high-dimensional linear space simplifies the prediction of chaotic motion in dense crowds. This approach bypasses the need for complex social interaction modeling by embedding interaction effects directly into the coordinate transformation.

Enforcing Mathematical Consistency and Stability

Simply training a neural network to find any linear representation can lead to practical failures, such as numerical divergence or overfitting to short-term transients. To combat this, researchers introduce structural constraints that respect the underlying physics of time.

For instance, the authors of Forecasting Sequential Data Using Consistent Koopman Autoencoders (N. Benjamin Erichson et al.) developed a framework that enforces discrete-space consistency between forward and backward time evolution. By ensuring that stepping forward in time and then backward in time returns the system to the exact same latent state, the model achieves superior stability and maintains high fidelity over extended prediction horizons, even in noisy environments.

Similarly, when dealing with highly chaotic or continuous-time systems, Ben Gao demonstrated in Conformal Online Learning of Deep Koopman Linear Embeddings that complex and chaotic dynamical systems can be accurately modeled using linear operators if the state is mapped into a sufficiently expressive high-dimensional representation. This allows the model to capture intricate behaviors like limit cycles and chaotic attractors with high precision.

Bridging Perception and Control

One of the most profound advantages of linearizing latent dynamics is that it unlocks decades of classical control theory for highly complex, high-dimensional systems.

In robotics and fluid mechanics, controlling a system typically requires knowing its exact nonlinear equations. By embedding these systems into a linear latent space, engineers can apply standard linear control techniques directly to the latent variables. For example, in Deep Dynamical Modeling and Control of Unsteady Fluid Flows, Jeremy Morton et al. applied this strategy to control unsteady fluid flows, bridging the gap between high-fidelity physical simulations and real-time control requirements.

This framework has also been successfully deployed in high-resolution tactile sensing. In TactileVAD: Geometric Aliasing-Aware Dynamics for High-Resolution Tactile Control, Miquel Oller et al. used globally linear latent dynamics to resolve geometric aliasing in robotic touch sensors, enabling the use of standard Linear Quadratic Regulators (LQR) to achieve stable and precise object manipulation.

Even in medical applications, such as closed-loop electrical neurostimulation for epilepsy, Zhichao Liang et al. demonstrated in Online Learning Koopman operator for closed-loop electrical neurostimulation in epilepsy that transforming chaotic brain signals into a linear latent space allows for real-time calculation of optimal stimulation pulses, suppressing seizures with massive computational efficiency compared to traditional neural network approaches.

Expanding to Sequence Models and Transformers

The principle of latent linearization is not limited to traditional physical systems; it is increasingly being used to stabilize and interpret modern deep learning architectures like Transformers.

Researchers have integrated Koopman-based linear layers into sequence models to handle non-stationary time series data. In Dynamics-aware Representation Learning via Multivariate Time Series Transformers, Michael Potter et al. constrained the latent space of a Transformer to follow linear time-invariant (LTI) dynamics, simplifying forecasting into a linear extension of learned temporal patterns.

To handle highly volatile, non-stationary data where statistical properties shift over time, the authors of VCformer: Variable Correlation Transformer with Inherent Lagged Correlation for Multivariate Time Series Forecasting (Yingnan Yang et al.) mapped time-series segments into a higher-dimensional measurement space where complex transitions are approximated linearly. This mechanism provides a more consistent modeling framework for highly volatile data, ensuring that the model remains robust even when the underlying statistical distributions evolve.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 762 papers
  2. Version 1 · 2026-10-04 · 442 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_koopman_operator_linearization_462d4597,
  title = {Koopman operator linearization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/koopman-operator-linearization/rdr_462d4597de554d45aa6754035b5069a2},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_koopman_operator_linearization_462d4597_v2,
  title = {Koopman operator linearization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/koopman-operator-linearization/rdr_462d4597de554d45aa6754035b5069a2/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake