Mori-Zwanzig mode decomposition: Comparison with time-delay embeddings

Michael Woodward, Yifeng Tian, Yen Ting Lin, Christoph Hader, Hermann F. Fasel, Daniel LivescuarXiv 2023 · DOI · arXiv

This paper introduces Mori-Zwanzig Mode Decomposition (MZMD), a new data-driven framework for analyzing complex, high-dimensional systems like fluid flows. By integrating classical statistical mechanics with modern modal analysis, the authors provide a more stable and efficient way to capture the "memory" of a system—the hidden influence of unresolved variables on the ones we can actually see.

TL;DR

MZMD extends the popular Dynamic Mode Decomposition (DMD) by adding "memory kernels" derived from the Mori-Zwanzig formalism. This allows it to capture nonlinear interactions and transient structures that standard methods miss, offering better stability and significantly lower computational costs than current time-delay embedding techniques.

What Problem Does This Paper Address?

In fluid dynamics and engineering, we often use Dynamic Mode Decomposition (DMD) to find dominant patterns (modes) in complex data. However, DMD has a major weakness: it assumes the system is linear. In reality, most interesting systems (like turbulent flows) are highly nonlinear.

When we only observe a subset of a system's variables, we run into the closure problem. The variables we don't see still influence the ones we do see. Standard DMD ignores this, often leading to "spurious" results or missing critical nonlinear interactions. Existing workarounds, like Higher-Order DMD (HODMD), try to fix this by stacking past data (time-delay embeddings), but these methods are computationally expensive, prone to overfitting, and can sometimes break the underlying mathematical structure of the physics.

Methods

The authors bridge the gap using the Mori-Zwanzig (MZ) formalism. Instead of just stacking data, MZMD models the system using the Generalized Langevin Equation (GLE). This equation decomposes dynamics into three parts:

  1. Markovian term: The immediate, instantaneous state (what DMD captures).
  2. Memory term: A convolution that accounts for how past states influence the present.
  3. Orthogonal dynamics: The "noise" or influence from unresolved scales.

What’s new: Unlike HODMD, which requires two separate, massive Singular Value Decomposition (SVD) steps, MZMD uses a recursive approach based on the Generalized Fluctuation-Dissipation (GFD) relation. This ensures the model remains physically consistent and computationally lean.

Datasets

The researchers evaluated MZMD against two Direct Numerical Simulation (DNS) datasets:

  • 2D Cylinder Flow (Re=200 & 600): A standard validation case involving periodic vortex shedding (the von Kármán vortex street).
  • Hypersonic Boundary-Layer Transition: A complex simulation of air flowing over a flared cone at Mach 6. This dataset is particularly challenging because it involves the transition from smooth (laminar) flow to chaos (turbulence) and the creation of "hot streaks" on the cone's surface.

Key Insights and Ideas

  • Memory as a "Closure" Term: The MZ memory kernels act as a mathematical "bridge," representing the energy and information flowing from unresolved scales back into the resolved ones.
  • Stability through Structure: MZMD explicitly preserves the "companion matrix" structure of the dynamics. The authors show that HODMD’s extra truncation step can actually break this structure, leading to numerical instability (the model "blowing up").
  • Physical Interpretability: In the Mach 6 case, MZMD identified specific transient modes that correspond to the nonlinear mechanisms creating surface hot streaks. DMD was unable to resolve these features.
  • Resistance to Overfitting: While HODMD often shows better "reconstruction" (fitting the training data), it frequently fails at "generalization" (predicting the future). MZMD showed a monotonic decay in its memory terms, a sign that it is capturing physical decay rather than just memorizing noise.

Key Results

  • Improved Accuracy: In the hypersonic transition case, MZMD reduced relative prediction error by approximately 3% compared to standard DMD.
  • Computational Efficiency: MZMD is nearly an order of magnitude cheaper than HODMD because it avoids the second large-scale SVD on massive Hankel matrices.
  • Frequency Precision: MZMD’s estimates for the primary instability frequencies were significantly closer to the ground-truth DNS data than either DMD or HODMD.
  • Stability: MZMD proved more robust, pulling unstable eigenvalues back toward the unit circle more effectively than competing methods.

Limitations and Future Work

The authors acknowledge that while MZMD is more stable, it does not yet guarantee long-term stability (eigenvalues can still occasionally drift into unstable territory). Furthermore, this implementation assumes the "noise" (orthogonal dynamics) is negligible, which may not hold for systems with extreme stochasticity.

Future directions include:

  • Integrating MZMD into control systems and online monitoring.
  • Exploring non-Gaussian noise modeling within the MZ framework.
  • Applying physics-informed machine learning to further constrain the MZ operators based on known physical laws (like conservation of energy).

Summary updated

Abstract

We introduce the Mori-Zwanzig Mode Decomposition (MZMD), a novel data-driven technique for efficient modal analysis of and reduced-order modeling of large-scale spatio-temporal dynamical systems. MZMD represents an extension of Dynamic Mode Decomposition (DMD) by providing an approximate closure term with MZ memory kernels accounting for how the unresolved modes of DMD interact with the resolved modes, thus addressing limitations when the state-space observables do not form a Koopman-invariant subspace. Leveraging the Mori-Zwanzig (MZ) formalism, MZMD identifies the modes and spectrum of the discrete-time Generalized Langevin Equation (GLE); an integro-differential equation that governs the dynamics of selected observables and their memory-dependent coupling with the unresolved degrees of freedom. This feature fundamentally distinguishes MZMD from time-delay embedding methods, such as Higher-Order DMD (HODMD). In this work, we derive and analyze MZMD and compare it with DMD and HODMD, using two exemplary Direct Numerical Simulation (DNS) datasets: a 2D flow over a cylinder (as validation) and laminar-turbulent boundary-layer transition over a flared cone at Mach 6. We demonstrate that MZMD, via the addition of MZ memory terms, improves the resolution of spatio-temporal structures within the transitional/turbulent regime by the introduction of transient and periodic modes (not captured by DMD), which contain features that arise due to nonlinear mechanisms, such as the generation of the so-called hot streaks on the surface of the flared cone. Our results demonstrate that MZMD serves as an efficient generalization of DMD (reducing to DMD in the absence of memory), improves stability, and exhibits greater robustness and resistance to overfitting compared to HODMD.
BibTeX
@misc{woodward2023mori,
  author = {Michael Woodward and Yifeng Tian and Yen Ting Lin and Christoph Hader and Hermann F. Fasel and Daniel Livescu},
  title = {{Mori-Zwanzig mode decomposition: Comparison with time-delay embeddings}},
  year = {2023},
  eprint = {2311.09524},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2311.09524},
}

Loading related content...

Figure