Gaussian Process Priors for Boundary Value Problems of Linear Partial Differential Equations

Jianlei Huang, Marc Harkonen, Markus Lange-Hegermann, Bogdan RaitaSubmission to ICLR 2026 · Rejected

TL;DR

Boundary Ehrenpreis–Palamodov Gaussian Processes (B-EPGPs) construct Gaussian process priors whose realizations analytically satisfy both linear constant-coefficient partial differential equations (PDEs) and linear boundary conditions. By baking boundary constraints directly into the GP basis functions via algebraic geometry and reflection groups, B-EPGP bypasses the curse of dimensionality on boundaries and improves accuracy and runtime over standard physics-informed baselines and neural operators.

What Problem Does This Paper Address?

While classical numerical solvers require well-posed PDEs with strictly defined boundary/initial conditions to compute unique solutions, machine learning methods (such as physics-informed neural networks or Gaussian processes) often tackle underdetermined or data-driven systems.

Previous methods like Ehrenpreis–Palamodov Gaussian Processes (EPGP) guarantee that GP sample paths are exact solutions to a linear PDE across the entire space. However, to handle boundary conditions, earlier methods had to condition the model on discrete observation points along the boundary. In higher dimensions, discretizing an interface requires $\mathcal{O}(g^{d-1})$ boundary grid points—a severe curse of dimensionality that causes memory blowups and inaccurate, non-physical boundary behavior (such as energy dissipation in wave equations).

Methods

B-EPGP combines the classical Ehrenpreis–Palamodov theorem with computational commutative algebra (Gröbner bases and syzygies) to build exact boundary-respecting basis functions:

  • Halfspaces & Syzygies: For a linear PDE $A(\partial)u = 0$ with boundary condition $B(\partial)u = 0$ on a hyperplane (e.g., $x_n = 0$), solutions are formed from exponential terms $e^{x \cdot z}$ on the characteristic variety $V = \{z \in \mathbb{C}^n : A(z) = 0\}$. By grouping frequencies that share the same tangential coordinates (the fiber $S_{z'}$), the boundary constraint reduces to finding weights $w_z$ such that $\sum_{z \in S_{z'}} w_z B(z) = 0$. These weights form a syzygy module, computable symbolically via Gröbner bases.
  • Polygonal Boundaries & Reflection Groups: For domains formed by intersecting halfspaces (wedges, slabs, boxes), boundary reflections generate reflection groups. When reflection orbits are finite (e.g., right-angle wedges), a finite combination of exponentials satisfies all boundaries simultaneously. When orbits are infinite (parallel boundaries), imposing boundary conditions naturally quantizes the frequencies, recovering Fourier sine/cosine series analytically.
  • Hybrid B-EPGP: For domains with curved boundaries (e.g., circular sectors), flat boundary pieces are enforced analytically via B-EPGP basis functions, while curved pieces are conditioned using sampled boundary data.
  • Inhomogeneous Systems: Inhomogeneous equations ($A(\partial)u = f, B(\partial)u = g$) are handled by computing a particular solution $u_p$ spectrally and using B-EPGP to model the remaining homogeneous error.

Datasets

The authors evaluate the method across several synthetic benchmark boundary value problems of varying geometry and dimension:

  • 1D Wave Equation: Neumann boundary condition on a half-line, initialized with Gaussian pulses.
  • 3D Wave Equation (4D space-time): Wave propagation with Neumann conditions on two orthogonal planes ($y=0, z=0$).
  • 2D Wave Equation in Varied Geometries: Rectangular domain, triangular domain, $45^\circ$ acute wedge, and a circular sector (hybrid flat/curved boundary).
  • 1D Wave in a Slab / Infinite Interval: Dirichlet and Neumann conditions on $[0, \pi]$.
  • 2D Heat Equation: Comparisons across free diffusion, Dirichlet, and Neumann boundaries in a wedge domain.
  • Singular Laplace Equation: 2D potential problem on domains adjacent to a singularity at the origin.

Key Insights and Ideas

  • Physics-Constrained vs. Physics-Informed: Instead of treating boundary conditions as penalty losses (like PINNs) or regression targets (like vanilla EPGP), B-EPGP enforces them identically in the prior’s function space.
  • Eliminating Boundary Discretization Cost: Because the boundary constraints are baked into the basis functions, no boundary collocation points are needed for flat/polygonal boundaries, eliminating the $\mathcal{O}(g^{d-1})$ boundary data requirement.
  • Rediscovering Separation of Variables Algebraically: Applying B-EPGP to parallel boundaries naturally yields eigenvalue quantization ($\sinh(b\pi) = 0 \implies b \in \sqrt{-1}\mathbb{Z}$), demonstrating that classical Fourier separation-of-variables falls out as a special case of this algebraic reflection framework.
  • Exact Energy Conservation: In wave equations, B-EPGP preserves the system's total physical energy over time, whereas baseline methods like EPGP dissipate energy due to imperfect boundary enforcement.

Key Results

  • Superiority over Neural Operators: On the 1D wave benchmark, B-EPGP achieved an absolute $L_1$ error of $3.41 \times 10^{-5}$ (0.06% relative error) at $n=1201$ points, outperforming Fourier Neural Operators (FNO, 1.17%) and Convolutional Neural Operators (CNO, 0.79%) by over an order of magnitude.
  • Outperforming Prior GP Methods: On the same 1D task, B-EPGP cut error roughly in half compared to BCGP ($6.34 \times 10^{-5}$), EPGP ($6.62 \times 10^{-5}$), and WIGPR ($8.34 \times 10^{-5}$). On a 2D halfspace wave problem, B-EPGP reduced median $L_1$ error by nearly $9\times$ ($0.48 \times 10^{-4}$ vs. $4.26 \times 10^{-4}$) and runtime by $>7.5\times$ (804s vs. 6059s) compared to EPGP.
  • Scaling to 4D Space-Time: For the 3D wave equation, baseline EPGP exhausted memory on an 80GB Nvidia A100 GPU due to boundary data density. B-EPGP successfully converged with a low $L_1$ error of $0.00088$.

Limitations and Future Work

  • Linear, Constant-Coefficient Systems Only: The mathematical foundation relies on the Ehrenpreis–Palamodov theorem, meaning B-EPGP cannot directly handle nonlinear PDEs or equations with spatially varying coefficients.
  • Curved Boundaries Require Sampling: Analytical basis construction is limited to planar/polygonal boundaries. Curved boundaries still require data-conditioning (Hybrid B-EPGP), retaining some sensitivity to sampling on those boundaries.
  • Optimization of Frequencies: While inference is fast once trained, optimizing the latent frequencies via marginal likelihood requires non-convex gradient descent, taking minutes to tens of minutes on higher-dimensional problems.

Summary updated

Abstract

Working with systems of partial differential equations (PDEs) is a fundamental task in computational science. Well-posed systems are addressed by numerical solvers or neural operators, whereas systems described by data are often addressed by PINNs or Gaussian processes. In this work, we propose Boundary Ehrenpreis--Palamodov Gaussian Processes (B-EPGPs), a novel probabilistic framework for constructing GP priors that satisfy both general systems of linear PDEs with constant coefficients and linear boundary conditions and can be conditioned on a finite data set. The Ehrenpreis--Palamodov theorem provides the functional form of the solution, but leaves parameters of this functional form unknown. An optimization finds these parameters and we provide an algorithm to modify this function form to satisfy boundary conditions. We explicitly construct GP priors for representative PDE systems with practical boundary conditions. Formal proofs of correctness are provided and empirical results demonstrating significant accuracy and computational resource improvements over state-of-the-art approaches.
BibTeX
@misc{huang2024gaussian,
  author = {Jianlei Huang and Marc Harkonen and Markus Lange-Hegermann and Bogdan Raita},
  title = {{Gaussian Process Priors for Boundary Value Problems of Linear Partial Differential Equations}},
  year = {2024},
  eprint = {2411.16663},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2411.16663},
}
Figure