Empirical sparse regression on quadratic manifolds

Paul Schwerdtner, Serkan Gugercin, Benjamin Peherstorfer

Abstract

Approximating high-dimensional field variables from sparse samples is a fundamental challenge in computational science and engineering. Traditional methods, such as gappy proper orthogonal decomposition (Gappy POD) and empirical interpolation, rely on linear approximation spaces. These methods often struggle with transport-dominated and wave-like dynamics due to the Kolmogorov barrier, where a large number of linear basis vectors are required to represent simple moving features. We introduce Quadratic Manifold Sparse Regression (QMSR), a framework that utilizes quadratic manifolds trained via a sparse greedy method. By employing novel nonlinear projections of sparse samples, QMSR achieves significantly higher accuracy and stability than linear methods, particularly for complex fluid dynamics and wave propagation.

Introduction

In many engineering applications, we must reconstruct a full state field—such as pressure across a turbine or temperature in an engine—using only a limited number of sparse sensors. For decades, the standard approach has been to project these sparse measurements onto a low-dimensional linear subspace. However, when the underlying physics involves moving shocks, traveling waves, or rotating structures, linear subspaces become inefficient. This inefficiency, often characterized as the Kolmogorov width problem, necessitates a high number of modes (and consequently, a high number of sensors) to maintain accuracy.

Quadratic Manifold Sparse Regression (QMSR) addresses this by allowing the approximation space to be a curved manifold rather than a flat plane, capturing the inherent nonlinearity of transport-dominated data with far fewer degrees of freedom.

Methodology

1. The Quadratic Decoder

QMSR represents the high-dimensional data $x$ using a quadratic expansion: $$x \approx Vs + Wh(s)$$ where $V$ is the linear basis, $s$ represents the reduced coordinates, and $Wh(s)$ is a nonlinear correction term. Here, $h(s)$ contains all unique quadratic combinations of the coordinates (e.g., $s_i^2$ and $s_i s_j$). This structure allows the model to represent "curved" trajectories in the high-dimensional space, which are typical of moving waves.

2. Sparse Linear Encoding

One of the primary innovations of QMSR is the discovery that finding the optimal coordinates on a nonlinear quadratic manifold does not necessarily require expensive nonlinear optimization. The authors demonstrate that a carefully constructed linear projection—specifically a Moore-Penrose pseudo-inverse applied to the sparse samples—is sufficient to accurately identify the state on the manifold. This maintains the computational efficiency required for real-time monitoring.

3. Sparse Greedy Training and Sensor Placement

To ensure stability, the manifold is not trained using standard global energy criteria (like SVD). Instead, a sparse greedy algorithm selects basis vectors that minimize reconstruction error specifically under sparse sampling conditions. Sensor locations are optimized using the Quality Discrete Empirical Interpolation Method (QDEIM) selection operator, ensuring that the chosen points provide the maximum information about the manifold's structure.

Experimental Results

The performance of QMSR was evaluated across three distinct numerical benchmarks:

  • Vlasov Equation: In simulating the density fields of collisionless charged particles, QMSR successfully captured the transport-dominated movement that typically causes linear methods to oscillate or require excessive modes.
  • Hamiltonian Pulse Signals: In 2D acoustic wave simulations, QMSR achieved up to six orders of magnitude higher accuracy than linear QDEIM. While linear methods exhibited instability when the number of sensors was close to the number of basis vectors, QMSR remained robust.
  • Rotating Detonation Waves (RDW): This high-fidelity dataset (1.7 million components per time step) represents the complex physics of next-generation propulsion. QMSR captured sharp shock fronts with high precision, whereas linear methods produced blurred reconstructions that failed to resolve the critical physics of the detonation front.

Discussion and Key Insights

The success of QMSR highlights that the bottleneck in field reconstruction is often the inflexibility of the linear model rather than a lack of sensor data. By moving to a quadratic manifold, the researchers reconstructed 360,000-dimensional fields from as few as 10 samples. Furthermore, the framework demonstrates that nonlinear manifolds can be effectively navigated using linear encoders, provided the manifold is structured correctly.

Limitations and Future Work

While quadratic manifolds offer a significant leap in performance, they remain a specific form of nonlinearity. Future research may explore more expressive feature maps, potentially learned via neural networks, to handle even more chaotic or discontinuous dynamics. Additionally, investigating the impact of sensor noise on the stability of the quadratic projection will be critical for transitioning this technology to real-world experimental environments.

Summary updated

Abstract

Approximating field variables and data vectors from sparse samples is a key challenge in computational science. Widely used methods such as gappy proper orthogonal decomposition and empirical interpolation rely on linear approximation spaces, limiting their effectiveness for data representing transport-dominated and wave-like dynamics. To address this limitation, we introduce quadratic manifold sparse regression, which trains quadratic manifolds with a sparse greedy method and computes approximations on the manifold through novel nonlinear projections of sparse samples. The nonlinear approximations obtained with quadratic manifold sparse regression achieve orders of magnitude higher accuracies than linear methods on data describing transport-dominated dynamics in numerical experiments.
BibTeX
@article{schwerdtner2025empirical,
  author = {Paul Schwerdtner and Serkan Gugercin and Benjamin Peherstorfer},
  title = {{Empirical sparse regression on quadratic manifolds}},
  journal = {SIAM J. Sci. Comput.},
  year = {2025},
  doi = {10.1137/24m1717270},
}
Figure