AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver

Xiangshu Gong, Zhiqiang Xie, Xiaowei Jin, Chen Wang, Yanling Qu, Wangmeng Zuo, Hui LiarXiv 2025 · DOI · arXiv

Researchers at the Harbin Institute of Technology have introduced a transformative approach to training AI for solving differential equations (DEs). By integrating a "reverse" data generation strategy with a novel Scale-Dilation Operator (SDO), they have developed a Transformer-based solver that significantly outperforms current state-of-the-art models in fluid dynamics, solid mechanics, and chaotic systems.

The Challenge: Data Scarcity and Spectral Bias

Solving DEs—the mathematical foundation of physics and engineering—traditionally involves a trade-off. Classical numerical solvers (e.g., Finite Element Methods) are accurate but computationally intensive. AI-based solvers offer speed but face two primary hurdles:

  1. Data Scarcity: Training requires massive datasets, typically generated by the very classical solvers the AI is meant to replace.
  2. Spectral Bias: Neural networks struggle to learn "high-frequency" components, such as sharp gradients or fine turbulent details, a problem known as the Approximation of High Frequency Components (AHFC).

Methodology: DE-Ruled Generation and SDO

The authors address these bottlenecks through a two-pronged innovation:

1. DE-Ruled Data Generation

Instead of the traditional "Forward" approach (starting with conditions and solving for the result), the researchers utilize an "Inverse-like" approach. They first generate a random or prior-knowledge-based solution field $u(t, x)$. By substituting this field into the target differential equation, they calculate the corresponding source terms ($f$), initial conditions ($u_0$), and boundary conditions ($g$). This allows for the creation of infinite, perfectly labeled training pairs at negligible computational cost.

Overview of the DE-ruled data generation process (a), turbulent flow fields at varying Reynolds numbers (b), and chaotic Lorenz system trajectories (c-d).

2. Scale-Dilation Operator (SDO)

To overcome spectral bias, the SDO uses Fourier transforms to "stretch" or dilate the solution in space and time. This dilation maps high-frequency components into a lower-frequency space, making them significantly easier for the neural network to optimize.

3. Architecture

The solver is a spatiotemporally coupled, attention-based Transformer. It processes spatial and temporal dimensions simultaneously, capturing complex dependencies in four-dimensional physical problems.

Evaluation and Datasets

The model was rigorously tested across five physical domains:

  • Navier-Stokes Equations: Including cylinder wakes and turbulent channel flows ($Re_\tau \sim 1000$).
  • Steady Navier–Cauchy (SNC): Linear elasticity in solids.
  • Wave Equations: Elastic wave propagation.
  • Equation of Motion (EoM): Structural dynamics using the Bouc–Wen model.
  • Forced Lorenz System: Testing temporal evolution in chaotic systems.

Key Insights

  • Intrinsic Mapping: The authors demonstrate that even if generated data represents non-physical scenarios, the AI learns the intrinsic mapping of the DE, which generalizes effectively to real-world physics.
  • Optimization Landscape: Mathematical proofs (Theorem 1) show that the SDO smooths the loss function's "terrain," preventing the model from failing to learn fine details.
  • Zero-Shot Generalization: The model accurately predicted flow around a complex leaf geometry—unseen during training—by mastering the underlying Navier-Stokes physics.

Results and Performance

  • Accuracy: The proposed model ("Ours w/ GenData") achieved an average error of 11%, vastly outperforming FNO (33%) and Transolver (35%).
  • Data Impact: Integrating DE-ruled data improved the performance of all baseline models (FNO, U-NO, etc.), reducing errors by 4% to 21%.
  • Turbulence Modeling: The model successfully simulated high-Reynolds number turbulent flows ($Re_\tau \sim 1000$), a traditionally difficult task for AI.

Limitations and Future Work

The "DE-ruled" generation currently relies on Fourier transforms, which are most efficient on regular grids. Adapting this to highly irregular, unstructured meshes remains a future challenge. Additionally, the computational scaling of the Transformer's attention mechanism for full-scale 3D engineering simulations (e.g., entire aircraft) requires further detailing.

This work marks a significant advancement toward making AI solvers practical for engineering by resolving the data bottleneck and the difficulty of capturing fine-scale physical details.

Summary updated

Abstract

Many problems are governed by differential equations (DEs). Artificial intelligence (AI) is a new path for solving DEs. However, data is very scarce and existing AI solvers struggle with approximation of high frequency components (AHFC). We propose an AI paradigm for solving diverse DEs, including DE-ruled first-principles data generation methodology and scale-dilation operator (SDO) AI solver. Using either prior knowledge or random fields, we generate solutions and then substitute them into the DEs to derive the sources and initial/boundary conditions through balancing DEs, thus producing arbitrarily vast amount of, first-principles-consistent training datasets at extremely low computational cost. We introduce a reversible SDO that leverages the Fourier transform of the multiscale solutions to fix AHFC, and design a spatiotemporally coupled, attention-based Transformer AI solver of DEs with SDO. An upper bound on the Hessian condition number of the loss function is proven to be proportional to the squared 2-norm of the solution gradient, revealing that SDO yields a smoother loss landscape, consequently fixing AHFC with efficient training. Extensive tests on diverse DEs demonstrate that our AI paradigm achieves consistently superior accuracy over state-of-the-art methods. This work makes AI solver of DEs to be truly usable in broad nature and engineering fields.
BibTeX
@misc{gong2025ai,
  author = {Xiangshu Gong and Zhiqiang Xie and Xiaowei Jin and Chen Wang and Yanling Qu and Wangmeng Zuo and Hui Li},
  title = {{AI paradigm for solving differential equations: first-principles data generation and scale-dilation operator AI solver}},
  year = {2025},
  eprint = {2507.23141},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.23141},
}
Figure