Degree-of-freedom and optimization-dynamic effects on the observability of Kuramoto-Sivashinsky systems

Nicolas Frank, Joshua L. Pughe-Sanford, Samuel J. GrauerarXiv 2025 · DOI · arXiv

This paper provides a rigorous look at the "how many sensors?" question for chaotic systems. By linking state estimation to embedding theory, the authors establish that the number of measurements required for observability is fundamentally tied to the dimension of the system's attractor. They also introduce a "pseudo-projection" technique that uses the system's own dynamics to keep numerical optimizers from drifting into unphysical, non-chaotic states.

TL;DR

How many sensors do you need to track a chaotic system? This paper uses embedding theory to prove that observability scales with the system's "effective degrees of freedom" (its inertial manifold dimension). It also introduces a "pseudo-projection" trick to keep optimizers from drifting into unphysical states during reconstruction.

What Problem Does This Paper Address?

Chaotic systems, from weather patterns to turbulent fluid flows, are famously sensitive to initial conditions—the "butterfly effect." To simulate them accurately, we use data assimilation (DA) to blend sparse sensor measurements with governing equations.

However, two major gaps persist:

  1. The Observability Gap: We don't have a clear rule for how many sensors (and at what frequency) are needed to uniquely determine a trajectory. Is one sensor enough if it samples fast? Do we need a grid?
  2. The Optimization Gap: Even with enough data, the "loss landscape" of a chaotic system is a nightmare. Standard optimizers often stall, get stuck in saddle points, or drift toward "unphysical" solutions that match the data points but violate the long-term behavior of the system.

Methods

The authors use the Kuramoto–Sivashinsky (KS) equation as their testbed—a classic 1D model for spatiotemporal chaos. Their approach combines high-level geometry with low-level optimization:

  • Embedding Theory: They apply Takens’ Theorem and its extensions to the system's Inertial Manifold (IM). This allows them to treat the chaotic attractor as a smooth, finite-dimensional surface ($d_M$) within an infinite-dimensional space.
  • 2DVar (Adjoint-Variational Estimation): They use a discrete adjoint framework to compute exact gradients and Hessians, parameterizing the entire trajectory by its initial condition.
  • Non-Convex Newton (NCN) Optimizer: To handle the "negative curvature" (regions where the landscape curves downward, confusing standard Newton methods), they use an NCN optimizer that regularizes the Hessian's eigenvalues.
  • Pseudo-Projection: This is the paper's cleverest trick. Periodically during optimization, they run the simulation forward for a very short burst. Because the system is dissipative, this acts as a "dynamical filter," pulling the estimate back toward the attractor and washing away unphysical high-frequency noise.

Datasets

The study relies on synthetic datasets generated by the authors:

  • Ground Truth: Long-time rollouts (10,000 time units) of the KS equation at three domain lengths ($L=22, 44, 66$).
  • Manifold Estimation: They trained Autoencoders on these rollouts to empirically estimate the dimension of the inertial manifold ($d_M$), finding $d_M$ values of 8, 18, and 28 respectively.
  • Test Cases: 8,000 reconstruction trials per domain size, varying sensor counts ($m_x$) and sampling rates ($m_t$).

Key Insights and Ideas

  • The $2d_M + 1$ Rule: The authors show that if your total number of measurements ($m$) is less than the manifold dimension ($d_M$), the system is locally unobservable. To guarantee global observability (meaning the only "perfect" solution is the true one), you generally need $m \ge 2d_M + 1$.
  • Drifting off the Manifold: A major cause of failure in DA isn't just "bad data," but the optimizer finding a trajectory that shadows the data while drifting into unphysical regions of state space. Pseudo-projection fixes this by using the physics as a stabilizer.
  • Destructive Interference: The paper suggests that when optimization stalls, it’s often not because of a local minimum, but because gradients from different time steps are "fighting" each other—a phenomenon they compare to multi-task learning conflicts.

Key Results

  • Theoretical Validation: Numerical experiments perfectly mirrored the embedding theory bounds. For $L=22$ ($d_M=8$), reconstruction success plummeted below $m=8$ and became robustly reliable only as $m$ approached 17.
  • Robustness: The NCN optimizer with pseudo-projection was able to recover trajectories even from "poor" initial guesses that were far from the true state.
  • Well-Posedness: They introduced a metric ($\epsilon^*$) to measure how "trustworthy" a low-loss solution is. They found that increasing the number of sensors doesn't just lower the error; it fundamentally changes the landscape to make the true solution easier to find.

Limitations and Future Work

  • Perfect Model Assumption: The study assumes the governing equations are known perfectly. In real-world engineering (like weather or flight), "model error" is a significant hurdle.
  • Noise-Free Data: While they touch on conditioning, the bulk of the results use clean synthetic data. Real sensors are noisy.
  • Scaling to 3D: The KS equation is 1D. Moving to 3D Navier-Stokes (turbulence) would involve manifold dimensions in the thousands or millions, requiring much more efficient ways to implement "pseudo-projection."
  • Multi-Task Optimization: The authors suggest that future optimizers should treat different observation times as "competing tasks" to prevent gradient cancellation.

Summary updated

Abstract

Simulations of chaotic systems can only produce high-fidelity trajectories when the initial and boundary conditions are well specified. When these conditions are unknown but measurements are available, adjoint-variational state estimation can reconstruct a trajectory that is consistent with both the data and the governing equations. A key open question is how many measurements are required for accurate reconstruction, making the full system trajectory observable from sparse data. We establish observability criteria for adjoint state estimation applied to the Kuramoto-Sivashinsky equation by linking its observability to embedding theory for dissipative dynamical systems. For a system whose attractor lies on an inertial manifold of dimension $d_M$, we show that $m \geq d_M$ measurements ensures local observability from an arbitrarily good initial guess, and $m \geq 2d_M + 1$ guarantees global observability and implies the only critical point on $M$ is the global minimum. We also analyze optimization-dynamic limitations that persist even when these geometric conditions are met, including drift off the manifold, Hessian degeneracy, negative curvature, and vanishing gradients. To address these issues, we introduce a robust reconstruction strategy that combines non-convex Newton updates with a novel pseudo-projection step. Numerical simulations of the Kuramoto-Sivashinsky equation validate our analysis and show practical limits of observability for chaotic systems with low-dimensional inertial manifolds.
BibTeX
@misc{frank2025degree,
  author = {Nicolas Frank and Joshua L. Pughe-Sanford and Samuel J. Grauer},
  title = {{Degree-of-freedom and optimization-dynamic effects on the observability of Kuramoto-Sivashinsky systems}},
  year = {2025},
  eprint = {2511.13643},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.13643},
  howpublished = {arXiv},
}
Figure