Empirical Sparse Regression on Quadratic Manifolds

Paul Schwerdtner, Serkan Gugercin, Benjamin Peherstorfer

TL;DR

Researchers have developed Quadratic Manifold Sparse Regression (QMSR), a new method to reconstruct high-dimensional data from just a few sensors. By ditching traditional linear subspaces for quadratic manifolds, QMSR achieves orders of magnitude higher accuracy on complex, "moving" data like waves and transport-dominated flows where standard methods typically fail.

What Problem Does This Paper Address?

In computational science, we often need to reconstruct a full "field" (like temperature across an entire engine) using only a handful of sparse measurements (sensors). The gold standard for this has long been methods like Gappy Proper Orthogonal Decomposition (Gappy POD) or Empirical Interpolation.

These methods rely on a fundamental assumption: the data lives in a linear subspace. However, many real-world phenomena—like a pulse traveling across a domain or a shock wave rotating in a turbine—don't fit well into linear boxes. This is known as the Kolmogorov barrier: to represent a moving wave linearly, you need a massive number of basis vectors, which in turn requires a massive number of sensors. If you don't have enough sensors, the reconstruction becomes unstable or wildly inaccurate.

Methods

The authors introduce QMSR, which replaces the linear approximation with a quadratic manifold.

  1. The Quadratic Decoder: Instead of just saying the data is a sum of basis vectors ($Vs$), they add a nonlinear correction term ($Wh(s)$), where $h(s)$ contains all the quadratic combinations of the reduced coordinates (e.g., $s_1^2, s_1s_2$, etc.). This allows the "space" to curve, capturing the movement of waves much more efficiently.
  2. The Sparse Encoder: To find the right coordinates from just a few samples, the authors use a "sparse linear encoder." Surprisingly, even though the manifold is nonlinear, a carefully constructed linear projection of the sparse samples is enough to find the right spot on that curved manifold.
  3. Sparse Greedy Training: You can't just pick the top-10 most energetic modes from an SVD (Singular Value Decomposition) and expect them to work with sparse sensors. The authors developed a greedy algorithm that picks basis vectors specifically to minimize reconstruction error under sparse sampling conditions.
  4. Sensor Placement: They use the QDEIM (Quality Discrete Empirical Interpolation Method) selection operator to decide where the "sensors" (sampling points) should go.

Datasets

The authors evaluated QMSR on three increasingly difficult numerical benchmarks:

  • Collisionless Charged Particles (Vlasov Equation): Simulates density fields of particles moving under an electric field. This is a classic "transport-dominated" problem.
  • Hamiltonian Interacting Pulse Signals: Simulates acoustic waves traveling and interacting in a 2D domain.
  • Rotating Detonation Waves (RDW): A high-fidelity dataset representing the complex physics (pressure, velocity, temperature) inside a next-generation propulsion engine. This is a massive dataset with over 1.7 million components per time step.

Key Insights and Ideas

  • Breaking the Kolmogorov Barrier: The "aha moment" is that the bottleneck in reconstruction isn't usually a lack of data points (sensors), but the inflexibility of the linear model. By allowing the model to be quadratic, the researchers could reconstruct 360,000-dimensional fields from as few as 10 samples.
  • Linear Encoders for Nonlinear Manifolds: The paper proves that you don't necessarily need a complex, computationally expensive nonlinear optimization to find the best fit on a quadratic manifold. A Moore-Penrose pseudo-inverse ($ (PV)^+ $) of the sparse samples works remarkably well.
  • Greedy Selection is Crucial: Simply taking the "best" global modes doesn't work for sparse sensing. You have to greedily pick modes that are "visible" and stable through the lens of your sparse sensors.

Key Results

  • Accuracy: In the Hamiltonian wave example, QMSR achieved up to six orders of magnitude higher accuracy than linear QDEIM when using the same number of sparse samples.
  • Stability: Linear methods often showed "instabilities" (huge error spikes) when the number of sensors was close to the number of basis vectors. QMSR remained stable and accurate.
  • Visual Fidelity: In the Rotating Detonation Engine test, linear methods produced blurry reconstructions that missed the sharp shock fronts. QMSR captured the shock fronts with high precision, looking almost identical to the "ground truth" full data.
  • Efficiency: QMSR with just $m=2r$ samples (where $r$ is the manifold dimension) performed nearly as well as if they had used the entire dataset to perform the reconstruction.

Limitations and Future Work

  • Quadratic Limit: While quadratic manifolds are a huge step up from linear ones, they are still a specific type of nonlinearity. The authors suggest that even more complex "feature maps" (perhaps learned via neural networks) could be used in the same framework.
  • Challenging Data: For the Rotating Detonation Wave dataset, while QMSR beat linear methods, the overall error was still higher than in the simpler wave cases. This suggests that extremely chaotic or discontinuous physics might require even more expressive manifolds.
  • Sensor Noise: The current study assumes "perfect" sparse samples. Future work could investigate how QMSR handles noisy sensor data, which is common in real-world engineering.

Summary updated

Abstract

Approximating field variables and data vectors from sparse samples is a key challenge in computational science. Widely used methods such as gappy proper orthogonal decomposition and empirical interpolation rely on linear approximation spaces, limiting their effectiveness for data representing transport-dominated and wave-like dynamics. To address this limitation, we introduce quadratic manifold sparse regression, which trains quadratic manifolds with a sparse greedy method and computes approximations on the manifold through novel nonlinear projections of sparse samples. The nonlinear approximations obtained with quadratic manifold sparse regression achieve orders of magnitude higher accuracies than linear methods on data describing transport-dominated dynamics in numerical experiments.
BibTeX
@article{schwerdtner2025empirical,
  author = {Paul Schwerdtner and Serkan Gugercin and Benjamin Peherstorfer},
  title = {{Empirical Sparse Regression on Quadratic Manifolds}},
  journal = {SIAM J. Sci. Comput.},
  year = {2025},
  doi = {10.1137/24m1717270},
}

Loading related content...

Figure