Introduction
Partial Differential Equations (PDEs) are the bedrock of modern science, modeling phenomena from heat diffusion to quantum mechanics. However, "learning" these equations from experimental data—reverse-engineering the underlying physics from observations—is a computationally grueling task. Standard approaches often require back-propagating gradients through simulations involving millions of interacting elements, leading to high memory and time costs.
While randomized algorithms have attempted to accelerate this process by exploiting sparsity, they are typically limited to systems with localized updates. In real-world scenarios, many physical systems exhibit "dense" updates—such as a laser heating a metal plate, causing temperature changes across the entire surface. This density breaks traditional compression methods, making the learning process slow and resource-intensive.
Methodology: The REEL Algorithm
The REEL (Random Projection based Efficient Learning) algorithm introduces a novel framework to accelerate PDE discovery by up to 98%. It achieves this by transforming complex physical updates into sparse components across both spatial and frequency domains, enabling high-fidelity learning in a compressed computational space.
1. Taylor Series Approximation
Many PDEs are non-decomposable, meaning their parameters and features are tangled in nonlinear ways. REEL uses Taylor expansions to approximate these nonlinear functions as polynomials. This "untangles" the equations into a decomposable form that is more amenable to dimensionality reduction via random projection.
2. Value and Frequency Domain Decomposition (VFDD)
The core innovation of REEL is the VFDD framework. Based on the Fourier uncertainty principle, a signal that is dense in the spatial domain (like a global temperature rise) is often sparse in the frequency domain. REEL splits system updates into two distinct parts:
- Value Component: Captures sharp, local changes, such as grain boundaries in a material.
- Frequency Component: Captures smooth, global changes, such as uniform heating.
3. Random Projection and Compressed Learning
Once the signals are decomposed into sparse components, REEL applies a random matrix to project the high-dimensional data into a much smaller space—sometimes just 1% of the original size. Because the signals are sparse, the essential physics is preserved. The optimization (via Stochastic Gradient Descent) then occurs entirely within this compressed space, drastically reducing computational overhead.
Experimental Results
The researchers evaluated REEL on two complex synthetic physics simulations: Solid-state Selective Laser Sintering (SLS) and Nanovoid Evolution.
Performance and Accuracy
REEL demonstrated massive speedups, reducing training time by 70% to 98% compared to non-compressed baselines. Remarkably, the algorithm maintained high accuracy even at 1% compression. In the nanovoid experiments, REEL successfully captured shrinking dynamics where previous state-of-the-art methods failed due to their inability to handle dense global updates.
Discussion and Future Work
REEL proves that the fundamental "physics" of a system often contains far less information than the raw data suggests. By leveraging dual-domain sparsity, REEL can discard 99% of the data while still accurately recovering the underlying laws of nature.
Future work will address the sensitivity of the decomposition threshold ($\beta$) and explore the algorithm's performance on noisy, real-world experimental data. Additionally, the impact of higher-order Taylor expansions on the balance between accuracy and computational speed remains an area for further investigation.
Summary updated