Meta-learning Pseudo-differential Operators with Deep Neural Networks

Jordi Feliu-Fabà, Yuwei Fan, Lexing YingJournal of Computational Physics 2019 · DOI · OpenReview

This research presents a clever intersection of classical numerical analysis and modern deep learning, introducing a "meta-learning" framework designed to solve complex physical equations by learning the underlying mathematical operators themselves.

Introduction

In scientific computing, many physical phenomena are modeled by equations of the form $L_\eta u = f$, where the operator $L_\eta$ depends on a parameter $\eta(x)$, such as a varying potential or scattering coefficient. Traditionally, solving for $u$ requires computing the inverse operator $G_\eta = L_\eta^{-1}$ (the Green's function) for every new $\eta$. This process is computationally intensive, particularly in high dimensions where $G_\eta$ becomes a massive, dense matrix.

This paper proposes a deep learning architecture that learns the mapping from the physical parameter $\eta$ to the solution operator $G_\eta$. By leveraging the nonstandard wavelet form to compress these operators, the network can be trained using only a few matrix-vector multiplications, effectively learning to "solve" entire classes of equations at once.

Methods: Operator Compression and Architecture

The core innovation is combining Deep Neural Networks (DNNs) with the nonstandard wavelet form. Pseudo-differential operators (PDOs), which include many common physical operators, are "sparse" when represented in a wavelet basis. The authors use this form to compress the dense operator $G_\eta$ into a small collection of vectors.

Nonstandard Wavelet Decomposition

The dense operator $A$ (representing $G_\eta$) is approximated by the product $W S W^T$, where $W$ represents the wavelet transform and $S$ is a structured, sparse matrix containing the wavelet coefficients.

Figure 3: Nonstandard wavelet decomposition of an operator A into the product of wavelet transform matrices W and a structured sparse matrix S.

The model consists of four main modules:

  1. $\eta \to S_\eta$ Module: A Convolutional Neural Network (CNN) that takes the parameter $\eta$ and predicts the compressed wavelet coefficients.
  2. Forward Wavelet Transform: A learnable module that transforms the input source term $f$ into the wavelet domain.
  3. Sparse Multiplication: A module that applies the predicted compressed operator to the transformed input.
  4. Inverse Wavelet Transform: A learnable module that brings the result back to the spatial domain to produce the solution $u$.

Algorithmic Implementation

The computation of $u = Av$ is performed efficiently using the steps outlined in Algorithm 1. This approach avoids the need to store or explicitly compute the full Green's function matrix, learning instead from pairs of $(\eta, \{f, u\})$.

Figure 4: Algorithm for computing the matrix-vector multiplication u = Av using the nonstandard wavelet transform.

Datasets and Evaluation

The authors evaluated their approach on several fundamental physical models, including the Schrödinger Equation (1D & 2D), Divergence Form Elliptic PDEs, and the Radiative Transfer Equation (RTE). The datasets were synthetically generated using traditional high-accuracy numerical solvers to provide ground-truth solutions for training.

Key Results and Insights

  • High Accuracy: The model achieved relative errors in the range of $10^{-2}$ to $10^{-3}$ across all test cases.
  • Efficiency: For the 1D Schrödinger case, the model represented the operator map with only ~30,000 parameters, a tiny fraction of what a dense representation would require.
  • Physics-Informed Compression: By using wavelets, the network only needs to learn the "important" parts of the operator. Furthermore, the network can "fine-tune" the wavelet transform itself to better suit the specific physics of the problem.
  • Generalization: Once trained, the model could instantly produce solutions for entirely new, unseen $\eta$ values with high precision, bypassing the need for expensive traditional solvers.

Limitations and Future Work

The current framework is specifically designed for linear operators $G_\eta$. Extending this to fully nonlinear PDEs is a major next step. Additionally, for problems with very long-range global interactions, the authors suggest incorporating multiscale neural networks or attention mechanisms. Adapting the wavelet-based compression to complex, non-rectangular boundaries also remains a challenge for future implementation.

Summary updated

Abstract

This paper introduces a meta-learning approach for parameterized pseudo-differential operators with deep neural networks. With the help of the nonstandard wavelet form, the pseudo-differential operators can be approximated in a compressed form with a collection of vectors. The nonlinear map from the parameter to this collection of vectors and the wavelet transform are learned together from a small number of matrix-vector multiplications of the pseudo-differential operator. Numerical results for Green's functions of elliptic partial differential equations and the radiative transfer equations demonstrate the efficiency and accuracy of the proposed approach.
BibTeX
@article{feliufaba2019meta,
  author = {Jordi Feliu-Fabà and Yuwei Fan and Lexing Ying},
  title = {{Meta-learning Pseudo-differential Operators with Deep Neural Networks}},
  journal = {Journal of Computational Physics},
  year = {2019},
  doi = {10.1016/j.jcp.2020.109309},
}
Figure