Abstract
Operator learning models like DeepONet and Fourier Neural Operators (FNO) have transformed the numerical solution of partial differential equations (PDEs). However, these models often struggle with "brittleness" when applied to target domains with different physical parameters than the training data. This paper presents FF-POD-DeepONet, a framework that integrates fusion frame theory and Fourier Feature Networks (FFNs) to enhance transferability. By utilizing redundant mathematical subspaces, the model achieves superior generalization and data efficiency in new physical scenarios.
Introduction
Partial Differential Equations (PDEs) are fundamental to modeling complex physical systems, describing phenomena from fluid dynamics to structural mechanics. While neural operators have shown promise in learning mappings between function spaces, their performance often degrades significantly when encountering shifts in domain parameters, such as changes in fluid viscosity or material properties. This "transfer learning" challenge typically necessitates large amounts of new data or extensive retraining, which is often impractical in real-world engineering.
To address this, we introduce FF-POD-DeepONet. This architecture leverages the redundancy of fusion frames to create a more robust representation of the underlying physics, allowing for high-accuracy predictions even with limited data from the target domain.
Methodology: FF-POD-DeepONet Architecture
The proposed FF-POD-DeepONet framework combines three key components: Fourier Feature Networks (FFNs), Fusion Frame Theory, and Proper Orthogonal Decomposition (POD).
- Fourier Feature Networks (FFNs): FFNs map input data into a high-dimensional spectral space, enabling the network to capture high-frequency patterns and intrinsic structures often missed by standard architectures.
- Fusion Frame Theory: Unlike traditional orthogonal bases, fusion frames allow for redundant, weighted subspaces. This redundancy provides flexibility and robustness against shifts in the underlying physical parameters, making the model less sensitive to domain shifts.
- Proper Orthogonal Decomposition (POD): Integrated within the DeepONet structure, POD identifies the most energetic modes of the functions, facilitating efficient dimensionality reduction within each fusion frame subspace.
The workflow involves partitioning the input space into multiple subspaces via FFNs, extracting features through POD, and processing these through parallel neural network branches. The final solution is obtained by fusing the outputs of these branches.
Experimental Evaluation
The framework was evaluated on three benchmark PDE problems: Darcy Flow, Burgers’ Equation, and an Elasticity model. These tests focused on both source domain performance and the ability to transfer to new domains with different parameters.
Darcy Flow and Transfer Performance
In modeling steady-state flow through porous media, the model was tested against shifts in Gaussian random field parameters and forcing terms. On the source dataset, FF-POD-DeepONet achieved a Mean Squared Error (MSE) of 0.0589, outperforming FNO (0.0671) and standard DeepONet (0.0803).
A key strength of the FF-POD-DeepONet is its performance in "few-shot" transfer scenarios. When adapting to a new target domain, the model maintains high accuracy with significantly fewer samples than competing methods.
As shown in the results for Darcy Flow transfer, FF-POD-DeepONet consistently yields the lowest MSE across all sample sizes, particularly excelling in the low-data regime (e.g., 20 samples), where it achieved an MSE of 0.2074 compared to 0.3025 for POD-DeepONet.
Discussion and Conclusion
The success of FF-POD-DeepONet highlights the value of mathematical redundancy in neural operator design. By capturing overlapping features through fusion frames, the model becomes less sensitive to variations in the target domain. Furthermore, the integration of FFNs allows for better sensitivity to the fine-grained physics essential for accurate transfer.
While the current results are promising, future work will explore automated hyperparameter tuning for subspace selection and scaling the framework to more complex, multi-scale physical problems beyond the 1D and 2D benchmarks presented here.
Reference: Jiang, H., & Qu, Y. (2024). Transfer Operator Learning with Fusion Frame. arXiv:2408.10458v1 [cs.LG].
Summary updated