Efficient Learning of PDEs via Taylor Expansion and Sparse Decomposition into Value and Fourier Domains

Md. Nasim, Yexiang Xue

Abstract

Learning the underlying Partial Differential Equations (PDEs) governing complex physical phenomena from experimental data is a computationally intensive task. We present REEL (Random projection based Efficient Learning), a framework that reduces training time by 70% to 98% without compromising accuracy. REEL achieves this through a novel Value and Frequency Domain Decomposition (VFDD) that exploits sparsity in both spatial and Fourier domains, combined with Taylor expansions to linearize non-decomposable physics. By training models on data compressed to 1% of its original volume via random projections, REEL facilitates faster scientific discovery in high-dimensional systems such as laser sintering and nanovoid evolution.

Introduction

The discovery of governing laws in the form of PDEs is fundamental to science and engineering. While deep learning has shown promise in learning these laws from data, the computational burden of back-propagating through large-scale simulations is significant. Previous acceleration methods, like the RAPID-PDE algorithm, rely on spatial sparsity—where changes occur only at specific interfaces. However, many physical processes, such as heat diffusion in laser sintering, involve "dense" updates across the entire domain, rendering purely spatial sparsity methods ineffective. REEL overcomes this by identifying sparsity in the frequency domain when spatial sparsity is absent.

Methodology

REEL addresses the challenges of high-dimensional physics learning through three integrated mathematical strategies:

1. Value and Frequency Domain Decomposition (VFDD)

Recognizing that signals dense in the spatial (value) domain are often sparse in the frequency domain, REEL decomposes updates into two components. A threshold $\beta$ separates high-frequency components (sparse in the value domain) from low-frequency components (sparse in the frequency domain). This dual-domain approach ensures that almost any physical update can be represented sparsely.

Overview of the VFDD process

2. Taylor Series Expansion

Non-linear PDEs often lack the decomposable structure required for efficient projection. REEL employs Taylor expansions to approximate these functions as polynomials, transforming complex physics into a format compatible with dimensionality reduction and "one-time" random projections.

3. Random Projection

The high-dimensional sparse signals are projected into a significantly smaller "compressed" space using a random matrix. Theoretical analysis confirms that minimizing loss in this projected space provides a constant factor approximation of the original problem, enabling massive speedups by training entirely within the compressed "shadow" space.

Experimental Setup

The framework was evaluated on two complex, high-dimensional physics simulations:

  • Solid-State Selective Laser Sintering (SLS): A 2D simulation of 3D printing involving phase-field (grain boundaries) and thermal (heat diffusion) models.
  • Nanovoid Evolution: A simulation of how nanovoids in materials evolve under extreme radiation and heat, a classic problem in nuclear materials science.

Results and Discussion

REEL demonstrated exceptional performance across all benchmarks, achieving training time reductions of 70% to 98% compared to standard baselines. Crucially, this speedup was achieved while training on data compressed to just 1% of its original size.

Training time and MSE comparison

As shown in the performance metrics, the training time for REEL remains nearly constant as the problem dimension increases, whereas the baseline scales poorly. Despite the extreme compression, the Mean Squared Error (MSE) for both density ($\phi$) and temperature ($T$) remains comparable to the baseline. In the nanovoid experiments, REEL successfully captured shrinking dynamics where previous state-of-the-art methods failed due to an inability to handle non-sparse background signals.

Conclusion

REEL provides a robust and efficient path for learning complex physics from data. While it requires careful selection of the decomposition threshold and Taylor expansion order, its ability to exploit dual-domain sparsity makes it widely applicable to systems that were previously computationally prohibitive. Future work will focus on automating hyperparameter selection and applying REEL to chaotic systems like turbulent fluid flow.

Summary updated

BibTeX
@inproceedings{nasim2024efficient,
  author = {Md. Nasim and Yexiang Xue},
  title = {{Efficient Learning of PDEs via Taylor Expansion and Sparse Decomposition into Value and Fourier Domains}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i13.29356},
}
Figure