Abstract
Deep Operator Networks (DeepONets) have emerged as efficient surrogate models for learning solution operators of partial differential equations (PDEs). However, recursive predictions using these models often suffer from "stability degradation," where errors accumulate over long time horizons. This paper introduces a transfer learning (TL) framework that addresses this issue by sequentially updating only the last layer of a pre-trained DeepONet using physics-informed losses. This method achieves stable, high-accuracy predictions for complex evolution equations, such as Navier-Stokes and Allen-Cahn, while requiring significantly less training data than standard approaches.
Introduction
Partial Differential Equations (PDEs) are fundamental to modeling physical systems, yet traditional numerical solvers are often computationally prohibitive for real-time applications. Deep learning models like DeepONet offer a faster alternative by learning the mapping from initial conditions to future states. Despite their potential, these models face a critical challenge: long-time instability. In recursive prediction settings, small errors at each step compound, leading to simulations that eventually diverge from physical reality. Furthermore, static networks often fail to adapt to the evolving physical complexity of dynamic systems.
Methodology
The proposed Transfer Learning (TL) aided DeepONet treats the network as an evolving operator rather than a static one. The architecture leverages the standard DeepONet structure, comprising a Branch net to encode initial conditions and a Trunk net for spatial coordinates.
The core innovation lies in the sequential updating process:
- Feature Extraction: The majority of the network, including the Trunk net and the hidden layers of the Branch net, is frozen after initial training to act as fixed feature extractors.
- Sequential Updating: At each prediction step, only the weights ($w$) of the last hidden layer in the Branch net are re-trained.
- Physics-Informed Loss: Updates are driven by a physics-informed loss function based on the PDE residual, eliminating the need for labeled ground-truth data during the prediction phase.
- Efficiency: By restricting optimization to the final linear layer, the update becomes a computationally inexpensive least-squares problem.
Experimental Evaluation
The framework was tested on several challenging evolution equations, including the Reaction-Diffusion equation, the Allen-Cahn and Cahn-Hilliard equations (notorious for sharp transition layers), and the 2D Navier-Stokes equation. It was also evaluated on the Multiscale Linear Radiative Transfer Equation (RTE), testing performance across different physical regimes.
Key Results and Insights
The TL-DeepONet demonstrates significant improvements in both stability and data efficiency. By viewing DeepONet as a learned Galerkin method—where the Trunk net provides basis functions and the Branch net provides coefficients—the transfer learning step allows these coefficients to adapt to changing solution regularity.
Key findings include:
- Long-Term Stability: In the 1D Allen-Cahn equation, TL-DeepONet maintained a relative $L^2$ error of approximately $10^{-3}$ up to $T=50$, whereas the vanilla DeepONet's error reached $10^0$ almost immediately.
- Fluid Dynamics: For 2D Navier-Stokes simulations, the TL approach reduced prediction errors by two orders of magnitude compared to standard models.
- Data Efficiency: A TL-DeepONet trained on 1,000 samples outperformed a standard model trained on 10,000 samples, highlighting its robustness with limited data.
Conclusion
The transfer learning framework effectively mitigates error accumulation in long-term PDE simulations. While the method relies on the initial training data being sufficiently diverse to "prime" the frozen Trunk net with necessary spatial features, it offers a powerful computational balance between pure forward passes and full retraining. Future work may explore more advanced time-discretization schemes and applications to high-Reynolds-number turbulent flows.
Summary updated