Neural operators have revolutionized scientific computing by learning mappings between continuous function spaces, enabling near-instantaneous predictions of complex physical systems. However, their practical deployment faces a severe bottleneck: data inefficiency. Traditional neural operators are purely data-driven, meaning they require massive datasets of high-fidelity simulations generated by expensive classical numerical solvers.
For instance, a study by Zachary E. Ross et al. in Deep neural Helmholtz operators for 3-D elastic wave propagation and inversion relied on 27,000 simulations to train a 3D wave propagation model. As noted by Jia Zhang et al. in LordNet: An efficient neural network for learning to solve parametric partial differential equations without simulated data, generating this volume of reference data is often as computationally expensive as the problem the neural network aims to accelerate. When training data is scarce, standard architectures suffer from severe performance degradation and fail to capture complex physical mappings (Yuan Qiu et al., Derivative-enhanced Deep Operator Network).
To overcome this, a major research direction focuses on sample-efficient operator learning. These methods drastically reduce the number of required high-fidelity training samples by embedding physical constraints, leveraging pre-training, and optimizing data acquisition.
Hardcoding Physics as Structural Inductive Biases
The most direct way to improve sample efficiency is to restrict the model's hypothesis space so it cannot learn physically impossible solutions. Instead of forcing a neural network to discover the laws of physics entirely from data, researchers embed mathematical structures directly into the network architecture or loss functions.
Consider how Green's functions describe the behavior of linear partial differential equations (PDEs) by mapping how a localized source propagates through a domain. Pietro Sittoni et al. exploit this in Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver by baking the hierarchical structure of Green's functions directly into the network layers using Hierarchical Semi-Separable (HSS) matrices. Because the network's internal matrix operations natively mirror the mathematical structure of the physical operator, it requires significantly fewer parameters and simulation samples to achieve high accuracy.
Symmetries offer another powerful inductive bias. If a physical system is invariant under certain transformations (like translation, scaling, or rotation), a model should respect these rules implicitly. Tara Akhound-Sadegh et al. introduce continuous Lie point symmetries into the training loop in Lie Point Symmetry and Physics-Informed Networks. By adding a custom symmetry loss, the model learns not just a single solution trajectory, but an entire family of valid neighboring solutions from a single data point, drastically boosting sample efficiency in low-data regimes.
Similarly, YanHui Zhu et al. address the optimization challenges of physics-informed models in APIC: Orthogonalized Neuro-Symbolic Modeling for Nonlinear Dissipative Dynamics. By isolating and orthogonalizing the gradients of a symbolic physics engine from the neural backbone, they eliminate the "tug-of-war" between data-driven learning and physical constraints, enabling accurate 3D fluid dynamics modeling with up to 90% less data. Other structural approaches, such as the spectral attention mechanism proposed by Han Wan et al. in PeSANet: Physics-encoded Spectral Attention Network for Simulating PDE-Governed Complex Systems, combine local differential operators with global frequency-domain features to accurately forecast complex systems using as few as 2 to 5 trajectories.
Transfer Learning and Foundation Models for Operators
In computer vision and natural language processing, massive pre-training allows models to adapt to downstream tasks with minimal data. A parallel paradigm is emerging in operator learning. Instead of training a neural operator from scratch for every new boundary condition or parameter set, models can be pre-trained on broad physical distributions.
Anthony Y. Zhou et al. provided a systematic study of these approaches in Strategies for Pretraining Neural Operators, demonstrating that while standard vision-inspired pre-training tasks fail, physics-based pre-training combined with spatial shift augmentations significantly mitigates data scarcity. Taking this to the scale of foundation models, Zhongkai Hao et al. developed DPOT: Auto-Regressive Denoising Operator Transformer for Large-Scale PDE Pre-Training. By utilizing an auto-regressive denoising pre-training strategy on a 1-billion-parameter Fourier-based transformer, the model achieves zero-shot generalization to entirely unseen physical tasks.
To bridge the generalization gap across completely different families of equations, Shandian Zhe et al. introduced Multi-Operator Few-Shot Learning for Generalization Across PDE Families. Their framework combines self-supervised spatial-frequency pre-training with text-based physical summaries, allowing the model to predict solutions for entirely new PDE families using only a handful of demonstration examples. Furthermore, Zecheng Zhang et al. demonstrate in DeepONet as a multi-Operator extrapolation model: Distributed pretraining with physics-Informed fine-Tuning that combining distributed pre-training with physics-informed fine-tuning allows neural operators to extrapolate effectively to new tasks without requiring extensive downstream data.
Intelligent Data Generation and Active Learning
When high-fidelity simulation data is unavailable, sample efficiency can be achieved by changing how and where data is collected. Rather than randomly sampling the parameter space, models can actively query data where it provides the highest information gain.
Alicja Polanska et al. implement this via physics-based active learning in Data-Efficient Neural Operator Training via Physics-Based Active Learning. Instead of relying on a fixed, pre-computed dataset, the training framework evaluates how well the neural operator satisfies the governing PDE's residual across the domain. It then intelligently selects new training points specifically where the residual error is highest, maximizing data efficiency by focusing computational resources on the hardest-to-learn physics.
Alternatively, the data bottleneck can be bypassed by generating training data analytically rather than numerically. In Data Generation-Based Operator Learning for Solving Partial Differential Equations on Unbounded Domains, Jihong Wang Jihong Wang et al. tackle the challenge of unbounded domains—where traditional numerical solvers struggle or fail. By generating approximate analytical solutions to construct the training set, they enable operator networks to learn complex mappings without requiring expensive pre-computed numerical simulations.
Architectural Adaptations for Low-Data Regimes
Beyond physics constraints and pre-training, the choice of core network architecture plays a crucial role in how efficiently a model processes limited data. For instance, Jules Berman introduces CoLoRA: Continuous low-rank adaptation for reduced implicit neural modeling of parameterized partial differential equations, which continuously adapts low-rank neural network weights over time. This approach yields significantly better data efficiency than standard neural operators, particularly for transport-dominated problems like fluid flows.
Similarly, alternative sequence-modeling backbones can offer better inductive biases for continuous dynamics than standard Transformers. Chun-Wun Cheng et al. explore this in Mamba Neural Operator: Who Wins? Transformers vs. State-Space Models for PDEs, establishing a theoretical link between State-Space Models (SSMs) and the iterative logic of neural operators. They demonstrate that the Mamba backbone maintains stability and accuracy in data-scarce regimes where traditional Transformers degrade.
Other frameworks, such as the meta-learning approach by Xiang Huang et al. (Meta-Auto-Decoder for Solving Parametric Partial Differential Equations), the hybrid sequence-modeling in PhysicsSolver by Zhenyi Zhu et al., and the hybrid quantum-classical framework proposed by Chanyoung Kim et al. (Quantum spectral operator learning for solving partial differential equations), all underscore a unified push toward architectures that natively minimize data dependency.
Key Foundational Work
Learning the solution operator of parametric partial differential equations with physics-informed DeepONets (2021) — contributed 1 concept elements to this direction.
Meta-Auto-Decoder for Solving Parametric Partial Differential Equations (2022) — contributed 1 concept elements to this direction.
Learning Green's functions associated with time-dependent partial differential equations (2022) — contributed 1 concept elements to this direction.