Data Efficient Surrogate Modeling for Physical Systems

87 concepts88 papers1 proposal

Deep learning has promised to revolutionize scientific computing by replacing slow numerical solvers—such as finite element methods or direct numerical fluid simulations—with fast neural network surrogates. However, this promise introduces a fundamental paradox: training a high-fidelity surrogate model typically requires a massive dataset of ground-truth simulations. If generating this training data takes longer than the total time saved by the surrogate during inference, the entire approach becomes computationally unviable.

To overcome this high-fidelity data bottleneck, researchers are developing methods for data-efficient surrogate modeling. Instead of treating the neural network as a blank slate trained on random simulation grids, this paradigm focuses on minimizing data acquisition costs through intelligent sampling, multi-fidelity fusion, structural inductive biases, and generative bootstrapping.


Intelligently Sampling the Parameter Space

The most direct way to combat data scarcity is to avoid running unnecessary simulations. Traditional datasets rely on random sampling or uniform grids, which waste immense computational budgets on simple or redundant regions of the parameter space while undersampling highly non-linear phenomena.

Active learning reframes data generation as an iterative, closed-loop process. Instead of pre-computing a static dataset, the surrogate model itself helps decide which simulation to run next. As demonstrated by Musekamp et al. in their comprehensive benchmark Active Learning for Neural PDE Solvers, guiding data acquisition via model uncertainty drastically reduces prediction errors and prevents catastrophic failures in worst-case scenarios.

[ Initial Small Dataset ] ──> [ Train Surrogate Model ]
          ▲                               │
          │                               ▼
[ Run Expensive Solver ] <── [ Select Most Uncertain Points ]

To maximize the efficiency of active learning, researchers are moving beyond selecting from a predefined pool of candidates:

  • Continuous Query Synthesis: Rather than picking from a discrete set of available points, Wang et al. developed a framework called PaPQS in A Plug-and-Play Query Synthesis Active Learning Framework for Neural PDE Solvers. It mathematically synthesizes the most informative training examples directly within the continuous parameter space, ensuring that every single call to an expensive partial differential equation (PDE) solver yields maximum information.
  • Dynamic Mesh Optimization: Data efficiency can also be achieved by optimizing the internal resolution of individual simulations. In Accelerating PDE Surrogates via RL-Guided Mesh Optimization, Meng et al. utilize reinforcement learning to adaptively allocate dense mesh points only where the underlying physics is complex (e.g., sharp gradients or turbulence), slashing data generation overhead by up to 50%.

Leveraging the Cheap to Learn the Expensive

In many scientific domains, we are not completely starved of data; rather, we are starved of high-fidelity data. Coarse-grid simulations or simplified physical models (low-fidelity) are computationally cheap and abundant, but they suffer from systematic biases. High-fidelity simulations are accurate but prohibitively expensive.

Multi-fidelity fusion works because low-fidelity models often capture the global, qualitative trends of a system correctly, even if their absolute values are wrong. By training a surrogate to learn the underlying relationship between low- and high-fidelity outputs, the model only needs a handful of high-fidelity data points to correct the bias and fill in fine-scale details.

Sahli Costabal et al. successfully applied this intuition in Multi-fidelity classification using Gaussian processes: Accelerating the prediction of large-scale computational models, demonstrating that multi-fidelity Gaussian processes can construct highly accurate classifiers with a fraction of the expensive data points normally required. When scaling this concept to high-dimensional physical fields, Wang et al. introduced Generalized Autoregression (GAR) in GAR: Generalized Autoregression for Multi-Fidelity Fusion, creating a mathematical framework capable of fusing multi-million-dimensional outputs even when the high-fidelity data points do not perfectly align with the low-fidelity sampling grid.


Baking Physics into Network Architecture

Standard neural networks are universal function approximators, meaning they must spend a massive amount of data just learning basic physical principles—such as spatial superposition, conservation laws, or localized interactions. If these mathematical principles are hardcoded directly into the neural network's architecture, the model's hypothesis space shrinks dramatically. It no longer needs data to discover the laws of physics; it only needs data to fit the specific boundary conditions of the problem.

A prominent example of this architectural constraint is found in DGNet: Discrete Green Networks for Data-Efficient Learning of Spatiotemporal PDEs. Here, Tan et al. explicitly embed Green’s function theory and the principle of linear superposition into the network layers. Because the model inherently understands how localized forces propagate through space, it achieves state-of-the-art accuracy using only a few dozen training trajectories.

Similarly, Sittoni et al. proposed Neural-HSS in Neural-HSS: Hierarchical Semi-Separable Neural PDE Solver, which leverages Hierarchical Semi-Separable matrices to mirror the hierarchical structure of Green's functions. This architectural bias yields extreme data efficiency, particularly in complex 3D simulations where generating ground-truth data is most punishing.


Bootstrapping with Generative and Semi-Supervised Loops

When labeled high-fidelity data cannot be expanded, researchers turn to semi-supervised learning and generative modeling to extract value from unlabeled parameter spaces or low-resolution data.

In Generative downscaling of PDE solvers with physics-guided diffusion models, Lu et al. combine the generative power of diffusion models with classical physics. Their framework, PGDM, uses a conditional diffusion model to generate a high-resolution "warm start" from low-resolution data, which is then refined using a few steps of a traditional numerical solver. This approach allows the model to produce high-fidelity results under severe data scarcity by trading pure data dependency for a small amount of runtime physics computation.

[ Low-Res Input ] ──> [ Conditional Diffusion Model ] ──> [ High-Res Warm Start ]
                                                                  │
                                                                  ▼
[ High-Fidelity Output ] <─────────────────────────────── [ Numerical Refinement ]

For purely data-driven operators, semi-supervised teacher-student frameworks offer an alternative path. In Uncertainty-Informed Meta Pseudo Labeling for Surrogate Modeling with Limited Labeled Data, Ren et al. introduced UMPL. This framework uses a teacher model to generate pseudo-labels for unlabeled physical configurations. To prevent the student model from learning incorrect physics, the authors implement a meta-learning loop driven by two distinct types of uncertainty (epistemic and aleatoric), allowing the model to safely exploit unlabeled data and achieve a ~14% performance boost when labeled data is critically scarce.


Foundation Models and Cross-Domain Transfer

Finally, data efficiency can be achieved by borrowing data from other domains. Just as large language models pretrain on massive text corpora before fine-tuning on specific tasks, scientific surrogates can pretrain on diverse, easily simulated physical systems before adapting to a data-poor target domain.

In Multiple Physics Pretraining for Spatiotemporal Surrogate Models, Bietti et al. demonstrated that pretraining a single model across a wide variety of fluid dynamics and wave equations allows it to build a generalized representation of spatiotemporal dynamics. When introduced to a new, data-scarce physical scenario, this pretrained "foundation model" requires significantly fewer high-fidelity samples to achieve high accuracy compared to a specialist model trained entirely from scratch.

This transfer capability is echoed in engineering design tasks as well. For instance, Hakhamaneshi et al. showed in Pretraining Graph Neural Networks for few-shot Analog Circuit Modeling and Design that pretraining a model on computationally cheap DC node voltages allows it to adapt to highly complex, data-scarce circuit performance metrics with a 10x improvement in sample efficiency.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_high_fidelity_data_scarcity_b10d96f6,
  title = {high-fidelity data scarcity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-fidelity-data-scarcity/rdr_b10d96f63b5c4db0ab49cb1d434ffc5e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_high_fidelity_data_scarcity_b10d96f6_v1,
  title = {high-fidelity data scarcity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/high-fidelity-data-scarcity/rdr_b10d96f63b5c4db0ab49cb1d434ffc5e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake