Mode Decomposition for Homogeneous Symmetric Operators

Ido Cohen, Omri Azencot, Pavel Lifshitz, Guy GilboaOpenReview 2020 · DOI · OpenReview

Abstract

Dynamic Mode Decomposition (DMD) is a widely used tool for extracting spatial structures and temporal dynamics from complex data. However, its application to nonlinear homogeneous gradient flows reveals a fundamental limitation termed the "DMD paradox." This paper demonstrates that standard DMD fails to capture the finite-time extinction characteristic of these flows due to its underlying linear exponential assumptions. To resolve this, we introduce Orthogonal Nonlinear Spectral decomposition (OrthoNS), a framework incorporating adaptive time-rescaling and symmetric DMD. OrthoNS successfully extracts nonlinear eigenfunctions and achieves reconstruction speeds 10 to 100 times faster than state-of-the-art methods.

Introduction

Extracting latent structures, or "modes," from high-dimensional data is essential in fields ranging from fluid dynamics to image processing. Dynamic Mode Decomposition (DMD) has emerged as a dominant algorithm for this purpose, typically modeling data as a sum of modes with exponential temporal dynamics. While effective for linear systems, DMD faces significant challenges when applied to nonlinear processes, particularly homogeneous gradient flows that minimize non-quadratic energies like the $p$-Dirichlet energy.

The DMD Paradox

Nonlinear homogeneous flows exhibit a unique behavior: they reach a steady state in finite time, following a polynomial decay profile rather than an exponential one. Standard DMD, which attempts to fit data to exponential curves, fundamentally fails to model this finite extinction. Even with infinitesimal sampling intervals, a strictly positive reconstruction error persists. This inherent mismatch is what the authors define as the DMD paradox.

Decay profile and spatial evolution of a nonlinear homogeneous flow. The plot (a) shows the polynomial decay $a(t)$, while (b)-(e) visualize the corresponding spatial field at different time steps $T$.

Methodology

To bridge the gap between linear DMD and nonlinear dynamics, the authors propose the OrthoNS framework, which consists of two primary components:

1. Time Re-scaling (Homogeneity Normalization)

The core innovation involves warping the temporal axis. By adaptively adjusting the time step size based on the flow's energy, the nonlinear polynomial decay is transformed into an exponential decay. This "homogeneity normalization" allows the linear DMD algorithm to perfectly model the underlying nonlinear system. A "blind" version of this rescaling is also provided for cases where the operator is unknown.

2. Symmetric DMD (S-DMD)

Since gradient flows are typically smoothing and non-oscillatory, their eigenvalues should be strictly real. Standard DMD does not guarantee this property. S-DMD modifies the optimization problem by constraining the DMD operator to be symmetric, ensuring physically consistent real eigenvalues.

Results and Discussion

The OrthoNS framework was validated using synthetic 1D signals and 2D images, including $p$-Laplacian eigenfunctions and natural images corrupted by noise.

  • Reconstruction Accuracy: Mathematical proofs and numerical experiments confirm that time-rescaled DMD achieves zero reconstruction error for flows initialized with a single eigenfunction.
  • Image Denoising: By decomposing noisy images into OrthoNS modes, the framework effectively performs edge-preserving denoising by filtering out high-eigenvalue modes associated with noise.
  • Computational Efficiency: OrthoNS demonstrates a massive performance advantage, operating 10 to 100 times faster than existing nonlinear $p$-spectral decomposition methods, with superior scaling for large datasets.

Conclusion

The DMD paradox highlights the limitations of rigid temporal sampling in analyzing nonlinear systems. By introducing time-warping and symmetry constraints, OrthoNS provides a powerful bridge between data-driven decomposition and nonlinear spectral theory. Future work will focus on addressing temporal error drift by optimizing reconstruction across the entire lifespan of the flow.

Summary updated

Abstract

Finding latent structures in data is drawing increasing attention in diverse fields such as image and signal processing, fluid dynamics, and machine learning. In this work we examine the problem of finding the main modes of gradient flows. Gradient descent is a fundamental process in optimization where its stochastic version is prominent in training of neural networks. Here our aim is to establish a consistent theory for gradient flows $\psi_t = P(\psi)$, where $P$ is a nonlinear homogeneous operator. Our proposed framework stems from analytic solutions of homogeneous flows, previously formalized by Cohen-Gilboa, where the initial condition $\psi_0$ admits the nonlinear eigenvalue problem $P(\psi_0)=\lambda \psi_0 $. We first present an analytic solution for \ac{DMD} in such cases. We show an inherent flaw of \ac{DMD}, which is unable to recover the essential dynamics of the flow. It is evident that \ac{DMD} is best suited for homogeneous flows of degree one. We propose an adaptive time sampling scheme and show its dynamics are analogue to homogeneous flows of degree one with a fixed step size. Moreover, we adapt \ac{DMD} to yield a real spectrum, using symmetric matrices. Our analytic solution of the proposed scheme recovers the dynamics perfectly and yields zero error. We then proceed to show that in the general case the orthogonal modes $\{ \phi_i \}$ are approximately nonlinear eigenfunctions $P(\phi_i) \approx\lambda_i \phi_i $. We formulate Orthogonal Nonlinear Spectral decomposition (\emph{OrthoNS}), which recovers the essential latent structures of the gradient descent process. Definitions for spectrum and filtering are given, and a Parseval-type identity is shown.
BibTeX
@misc{cohen2020mode,
  author = {Ido Cohen and Omri Azencot and Pavel Lifshitz and Guy Gilboa},
  title = {{Mode Decomposition for Homogeneous Symmetric Operators}},
  year = {2020},
  doi = {10.13140/rg.2.2.22624.10243},
  howpublished = {OpenReview},
}

Loading related content...

Figure