An Analysis of Constraint-Relaxation in PDE-Based Inverse Problems

Tristan van Leeuwen, Yunan Yang

TL;DR

This paper provides a rigorous mathematical analysis of "constraint-relaxation" for PDE-based inverse problems, showing that turning "hard" physical constraints into "soft" penalties can transform highly non-convex optimization landscapes into much friendlier, sometimes even convex, problems. The authors demonstrate that this approach reduces to a weighted least-squares problem where the weights can occasionally be derived directly from the measured data.

What Problem Does This Paper Address?

In science and engineering, we often face inverse problems: we have measurements (like seismic waves or medical scans) and want to figure out the physical properties (like rock density or tissue type) that produced them. Mathematically, this is usually framed as a PDE-constrained optimization problem.

The "hard" version of this problem requires the physics (the Partial Differential Equation, or PDE) to be satisfied exactly at every step of the optimization. This creates a "rugged" objective function—a "misfit landscape" full of local minima. If your initial guess is slightly off, your optimization algorithm gets "trapped," a phenomenon known in seismology as "cycle skipping." This paper explores a "relaxed" version where we allow the physics to be slightly "wrong" during the search, penalizing the error rather than forbidding it.

Methods

The authors move beyond empirical observations to provide a general framework for analyzing this relaxation. Their key methodological contributions include:

  1. The Representer Theorem: They prove that for a finite number of measurements, the estimated state of the system lives in a finite-dimensional subspace. This allows them to simplify a complex infinite-dimensional problem into a manageable finite-dimensional one.
  2. Variable Metric Reduction: They show that the relaxed problem is equivalent to a non-linear least-squares problem equipped with a specific, parameter-dependent weight matrix (a Gram matrix).
  3. Asymptotic Analysis: They analyze the limits of the penalty parameter ($\rho$).
  4. As $\rho \to \infty$, you recover the standard, difficult "hard-constrained" problem.
  5. As $\rho \to 0$, the problem focuses on the "PDE-residual," which the authors prove can become convex under certain conditions.
  6. Data-Driven Kernels: For specific cases like the Helmholtz and Schrödinger equations, they show the weight matrix can be estimated directly from the data, bypassing the need for some expensive PDE simulations.

Datasets

The paper uses several synthetic benchmark problems to validate the theory:

  • 1D and 2D Elliptic Equations: Used to visualize the optimization landscape.
  • 1D Helmholtz Equation: Used to demonstrate the data-driven kernel approach.
  • Schrödinger Equation: Used to show how a "direct method" can solve for coefficients using a linear system.
  • 2D Seismic Inversion: A realistic test case involving a 3x11 km domain with 124 sources/receivers to test robustness against poor initial guesses.

Key Insights and Ideas

  • The "Aha" Moment on Convexity: The most striking insight is that while the standard objective function is notoriously non-convex, the relaxed version (specifically the $\rho \to 0$ limit) can be strictly convex. This means there is only one "bottom of the bowl," making it impossible for local optimization to get stuck.
  • Interpolation as a Strategy: The penalty parameter $\rho$ acts as a bridge. By starting with a small $\rho$ (convex but perhaps physically loose) and increasing it, one can navigate the optimization toward the true physical solution without getting trapped.
  • Physics in the Weights: The weight matrix $G(c)$ isn't just a mathematical convenience; it represents the "Gram matrix" of the solutions. The fact that this can sometimes be built from data suggests a deep link between classical PDE optimization and modern data-driven reduced-order models.

Key Results

  • Landscape Smoothing: Figures 3 and 4 in the paper provide clear visual evidence: as the relaxation increases, the objective function transforms from a multi-modal mess into a smooth, convex curve.
  • Seismic Robustness: In the 2D seismic inversion tests, the conventional method failed to recover the true velocity model when given a poor starting guess (getting trapped in a local minimum). In contrast, the relaxed formulations (both variable-metric and data-driven) successfully recovered the deeper structures of the model.
  • Computational Efficiency: The authors show that the data-driven metric (Equation 24) achieves similar results to the more complex variable metric but at a lower computational cost, as it doesn't require differentiating the Gram matrix.

Limitations and Future Work

  • Scaling to 3D: While the theory is general, the authors acknowledge that for massive 3D problems, the matrix defining the metric will need to be approximated to remain computationally feasible.
  • Noise Sensitivity: The "data-driven" kernel estimation relies on clean data and derivatives with respect to parameters like frequency. How this holds up under significant experimental noise remains an open question.
  • Theoretical Bounds: The authors suggest that more work is needed to provide stronger statements on exactly when and why the relaxed problem remains convex in more complex, non-linear cases.

Summary updated

Abstract

Inverse problems are ubiquitous in science and engineering. Many of these are naturally formulated as a PDE-constrained optimization problem. These non-linear, large-scale, constrained optimization problems know many challenges, of which the inherent non-linearity of the problem is an important one. In this paper, we focus on a relaxed formulation of the PDE-constrained optimization problem and provide an in-depth analysis of it. Starting from an infinite-dimensional formulation of the inverse problem with discrete data, we propose a general framework for the analysis and discretisation of such problems. The relaxed formulation of the PDE-constrained optimization problem is shown to reduce to a weighted non-linear least-squares problem. The weight matrix turns out to be the Gram matrix of solutions of the PDE and, in some cases, can be estimated directly from the measurements. The latter observation points to a potential way to unify recently proposed data-driven reduced-order models for inverse problems with PDE-constrained optimization. We provide a number of representative case studies and numerical examples to illustrate our findings.
BibTeX
@inproceedings{leeuwen2025analysis,
  author = {Tristan van Leeuwen and Yunan Yang},
  title = {{An Analysis of Constraint-Relaxation in PDE-Based Inverse Problems}},
  booktitle = {Inverse},
  year = {2025},
  doi = {10.1088/1361-6420/adab89},
}
Figure