Learning Green's functions associated with parabolic partial differential equations

Nicolas Boullé, Seick Kim, Tianyi Shi, Alex TownsendOxford 2022 · OpenAlex

Abstract

Recent advancements in Scientific Machine Learning (SciML) have popularized the use of neural operators to solve partial differential equations (PDEs). However, the theoretical underpinnings of these models, particularly for time-dependent systems, have remained elusive. This paper establishes a rigorous framework for learning the solution operators of parabolic PDEs by approximating their underlying Green’s functions. By exploiting hierarchical low-rank structures and utilizing randomized numerical linear algebra, the authors provide the first theoretical guarantees for the sample complexity and learning rates associated with these physical systems.

1. Introduction

In the domain of SciML, the goal is often to learn a mapping between an input forcing term $f$ and the resulting solution $u$ of a PDE. While empirical successes are numerous, understanding the data requirements and approximation limits is critical for reliable engineering applications. Parabolic PDEs, which describe phenomena such as heat diffusion and fluid flow, present unique challenges due to their transient nature and the presence of singularities in their solution kernels, known as Green’s functions.

2. The Challenge of Parabolic Systems

Traditional machine learning theory often relies on $L^2$ (mean squared error) analysis. However, for parabolic PDEs in spatial dimensions $n > 1$, the Green’s function $G$ is typically not square-integrable due to singularities along the diagonal where space and time coordinates coincide. This makes standard Hilbert space techniques inapplicable. Furthermore, the temporal dimension introduces a dynamic component that requires a sophisticated spatio-temporal analysis rather than a static spatial one.

3. Methodology

The proposed framework bridges numerical analysis and machine learning through several innovative techniques:

3.1 Green’s Function Representation

The solution $u$ to a linear parabolic PDE can be represented as the integral transform of the forcing term $f$ with the Green’s function $G$: $$u(x, t) = \int_0^t \int_\Omega G(x, t, y, \tau) f(y, \tau) dy d\tau$$ By learning $G$ directly, the model effectively captures the entire solution operator rather than a single instance of the solution.

3.2 Hierarchical Low-Rank Partitioning

Although $G$ is singular on the diagonal, it exhibits a low-rank structure on "well-separated" domains. The authors introduce a hierarchical partitioning strategy (similar to an octree) that respects the parabolic scaling of the physics: $t \sim x^2$. This means that for a given spatial diameter, the temporal diameter of a subdomain is scaled quadratically, allowing for optimal compression of the operator across space and time.

3.3 Randomized Numerical Linear Algebra

To learn the low-rank blocks of the Green’s function without exhaustive sampling, the framework employs a randomized Singular Value Decomposition (SVD) for Hilbert-Schmidt operators. The operator is "probed" using random forcing functions drawn from a Gaussian Process (GP). The resulting solutions provide sufficient information to reconstruct the low-rank components of $G$ with high probability.

3.4 $L^1$-Norm Error Analysis

To address the lack of $L^2$ integrability in higher dimensions, the authors perform error analysis in the $L^1$-norm. Using Moser’s local maximum estimates, they bound the approximation error, providing a robust metric for convergence even when the Green's function blows up at the diagonal.

4. Complexity and Learning Rates

A central contribution of this work is the derivation of an intrinsic learning rate. The number of training pairs $N$ required to achieve an approximation error $\epsilon$ is shown to be: $$N = O(\epsilon^{-\frac{n+2}{2}} \log(1/\epsilon))$$ This formula quantifies the "curse of dimensionality" for PDE learning, showing how the spatial dimension $n$ influences the data requirements. The resulting approximant $\tilde{G}$ achieves a relative error of $O(\Gamma_\epsilon^{-1/2} \epsilon)$ in the $L^1$-norm, where $\Gamma_\epsilon$ represents the quality of the training data coverage provided by the Gaussian Process.

5. Discussion and Limitations

While this framework provides a solid theoretical foundation, certain practical challenges remain. The current algorithm requires the ability to evaluate the adjoint of the parabolic operator, which may be difficult in specific inverse problems or backward-in-time scenarios. Additionally, the low-rank assumptions do not extend easily to hyperbolic PDEs (e.g., the wave equation), where singularities propagate along characteristic lines rather than remaining localized on the diagonal, making them significantly harder to compress.

6. Conclusion

This work provides the first theoretically guaranteed algorithm for learning the solution operators of time-dependent parabolic PDEs. By aligning the hierarchical structure of the learning algorithm with the underlying physics of the Green’s function, the authors demonstrate that complex physical systems can be learned with nearly optimal sample complexity. This paves the way for more principled designs of "physics-informed" neural architectures that explicitly incorporate hierarchical, low-rank structures.

Summary updated

Abstract

Given input-output pairs from a parabolic partial differential equation (PDE) in any spatial dimension n≥1, we derive the first theoretically rigorous scheme for learning the associated Green's function G. Until now, rigorously learning Green's functions associated with parabolic operators has been a major challenge in the field of scientific machine learning because G may not be square-integrable when n>1, and time-dependent PDEs have transient dynamics. By combining the hierarchical low-rank structure of G together with the randomized singular value decomposition, we construct an approximant to G that achieves a relative error of O(Γ−1/2ϵϵ) in the L1-norm with high probability by using at most O(ϵ−n+22log(1/ϵ)) input-output training pairs, where Γϵ is a measure of the quality of the training dataset for learning G, and ϵ>0 is sufficiently small. Along the way, we extend the low-rank theory of Bebendorf and Hackbusch from elliptic PDEs in dimension 1≤n≤3 to parabolic PDEs in any dimensions, which shows that Green's functions associated with parabolic PDEs admit a low-rank structure on well-separated domains.
BibTeX
@inproceedings{boulle2022learning,
  author = {Nicolas Boullé and Seick Kim and Tianyi Shi and Alex Townsend},
  title = {{Learning Green's functions associated with parabolic partial differential equations}},
  booktitle = {Oxford},
  year = {2022},
}

Loading related content...

Figure