Abstract
Learning the underlying Partial Differential Equations (PDEs) governing complex physical phenomena from experimental data is a computationally intensive task. We present REEL (Random projection based Efficient Learning), a framework that reduces training time by 70% to 98% without compromising accuracy. REEL achieves this through a novel Value and Frequency Domain Decomposition (VFDD) that exploits sparsity in both spatial and Fourier domains, combined with Taylor expansions to linearize non-decomposable physics. By training models on data compressed to 1% of its original volume via random projections, REEL facilitates faster scientific discovery in high-dimensional systems such as laser sintering and nanovoid evolution.
Introduction
The discovery of governing laws in the form of PDEs is fundamental to science and engineering. While deep learning has shown promise in learning these laws from data, the computational burden of back-propagating through large-scale simulations is significant. Previous acceleration methods, like the RAPID-PDE algorithm, rely on spatial sparsity—where changes occur only at specific interfaces. However, many physical processes, such as heat diffusion in laser sintering, involve "dense" updates across the entire domain, rendering purely spatial sparsity methods ineffective. REEL overcomes this by identifying sparsity in the frequency domain when spatial sparsity is absent.
Methodology
REEL addresses the challenges of high-dimensional physics learning through three integrated mathematical strategies:
1. Value and Frequency Domain Decomposition (VFDD)
Recognizing that signals dense in the spatial (value) domain are often sparse in the frequency domain, REEL decomposes updates into two components. A threshold $\beta$ separates high-frequency components (sparse in the value domain) from low-frequency components (sparse in the frequency domain). This dual-domain approach ensures that almost any physical update can be represented sparsely.
2. Taylor Series Expansion
Non-linear PDEs often lack the decomposable structure required for efficient projection. REEL employs Taylor expansions to approximate these functions as polynomials, transforming complex physics into a format compatible with dimensionality reduction and "one-time" random projections.
3. Random Projection
The high-dimensional sparse signals are projected into a significantly smaller "compressed" space using a random matrix. Theoretical analysis confirms that minimizing loss in this projected space provides a constant factor approximation of the original problem, enabling massive speedups by training entirely within the compressed "shadow" space.
Experimental Setup
The framework was evaluated on two complex, high-dimensional physics simulations:
- Solid-State Selective Laser Sintering (SLS): A 2D simulation of 3D printing involving phase-field (grain boundaries) and thermal (heat diffusion) models.
- Nanovoid Evolution: A simulation of how nanovoids in materials evolve under extreme radiation and heat, a classic problem in nuclear materials science.
Results and Discussion
REEL demonstrated exceptional performance across all benchmarks, achieving training time reductions of 70% to 98% compared to standard baselines. Crucially, this speedup was achieved while training on data compressed to just 1% of its original size.
As shown in the performance metrics, the training time for REEL remains nearly constant as the problem dimension increases, whereas the baseline scales poorly. Despite the extreme compression, the Mean Squared Error (MSE) for both density ($\phi$) and temperature ($T$) remains comparable to the baseline. In the nanovoid experiments, REEL successfully captured shrinking dynamics where previous state-of-the-art methods failed due to an inability to handle non-sparse background signals.
Conclusion
REEL provides a robust and efficient path for learning complex physics from data. While it requires careful selection of the decomposition threshold and Taylor expansion order, its ability to exploit dual-domain sparsity makes it widely applicable to systems that were previously computationally prohibitive. Future work will focus on automating hyperparameter selection and applying REEL to chaotic systems like turbulent fluid flow.
Summary updated