Extrapolating Beyond Training Data with Physics Informed Models

250 concepts225 papers2 proposals

Deep learning models are statistical chameleons. They excel at interpolation—fitting a smooth mathematical curve through dense clusters of training data. However, when forced to extrapolate into out-of-distribution (OOD) regimes, they fail catastrophically. For instance, a neural network trained to control a robot at low speeds will often hallucinate physically impossible trajectories when tested at high velocities. This happens because purely data-driven models capture localized statistical correlations rather than the underlying causal mechanisms of the system.

Physics-informed machine learning solves this fundamental limitation by anchoring neural networks to the invariant principles of nature. While data distributions shift depending on the environment, the laws of thermodynamics, gravity, and conservation remain constant. By embedding these laws directly into the learning process, researchers transform machine learning from an unconstrained curve-fitting exercise into a bounded optimization problem.

Understanding why this approach enables robust out-of-distribution generalization requires looking at three distinct mechanisms: architectural constraints, physics-informed loss functions, and residual gray-box frameworks.


Architectural Constraints: Restricting the Hypothesis Space

The most robust way to ensure a model generalizes out-of-distribution is to make it mathematically impossible for the model to violate physical laws. Standard neural networks possess a massive hypothesis space, meaning they can represent an infinite variety of functions—including those that violate reality. Architectural grounding prunes this hypothesis space, forcing the network's forward pass to flow through explicit physical equations.

Purely Data-Driven Model:
Input ───> [ Unconstrained Neural Network ] ───> Output (Can be physically impossible)

Architecturally Constrained Model:
Input ───> [ Neural Network ] ───> [ Differentiable Physics Layer ] ───> Valid Output
                                        (Euler-Lagrange / Newton-Euler)

Consider a robotic manipulator learning its own dynamics. Michael Lutter et al. designed Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning (DeLaN) to embed the Euler-Lagrange equations directly into the network's structure. Instead of predicting forces directly, the network learns to output mass matrices and potential energies, which are then passed through the analytical equations of motion. Because the network must compute forces through the lens of classical mechanics, it accurately predicts the forces required for high-velocity movements even when trained exclusively on low-speed data.

Similarly, Giovanni Sutanto et al. demonstrated in Encoding Physical Constraints in Differentiable Newton-Euler Algorithm that implementing the Recursive Newton-Euler Algorithm as a differentiable computational graph ensures positive mass and satisfies the triangle inequality for inertia.

By structuring the network's internal operations to mirror physical structures—such as incorporating kinematic tree structures in Graph-Operator World Models for Morphology-Parameter Generalization in Continuous Control—the model is prevented from hallucinating non-physical shortcuts during exploration. The physics acts as a hard guardrail, ensuring that extrapolation remains stable far outside the training distribution.


Physics-Informed Loss Functions: Soft Guardrails for Complex Fields

When physical laws cannot be easily hardcoded into a network's architecture, they can instead be enforced via the loss function. This approach treats physical principles as regularizers, penalizing the model whenever its predictions deviate from known differential equations or conservation laws.

In structural engineering, predicting how components respond to stress under novel geometries is notoriously difficult. Gautam Sreekumar et al. addressed this in Physics Informed Neural Network for Dynamic Stress Prediction by embedding the fundamental equations of motion into the loss function of a neural network. When the model is queried on entirely new structural configurations or dynamic loading sequences, the physics-informed loss prevents it from drifting into arbitrary statistical regressions, maintaining high accuracy without requiring exhaustive finite element simulations for every new geometry.

This soft-constraint mechanism is equally vital when training reinforcement learning agents within surrogate environments. As Julen Cestero et al. showed in Optimizing energy management of smart grid using reinforcement learning aided by surrogate models built using physics-informed neural networks, purely data-driven surrogates fail when an agent explores unmapped regions of a smart grid, producing non-physical states that break the training loop. Incorporating power flow equations into the loss function allows the surrogate to extrapolate reliably, preventing the agent from learning invalid behaviors.

Whether dealing with wave propagation in seismic imaging, as explored by Robert W. Clayton et al. in Ambient Noise Full Waveform Inversion with Neural Operators, or fluid dynamics and convection terms designed by Pengwei Liu in PAPM: A Physics-aware Proxy Model for Process Systems, physics-informed losses ensure that even when the model encounters out-of-distribution inputs, its predictions remain anchored to scientific reality.


The Residual Framework: Graceful Fallbacks to Baseline Physics

Perhaps the most intuitive reason why physics-informed models excel at out-of-distribution generalization is the residual learning (or gray-box) framework. Instead of demanding that a neural network learn a complex physical system entirely from scratch, researchers use a trusted, analytical physics model as a baseline foundation. The neural network is then tasked solely with predicting the residual—the unmodeled errors, friction, or discrepancies between the simplified physics equations and the real world.

$$\text{Total Prediction} = \text{Physics Baseline}(x) + \text{Neural Network Residual}(x)$$

Leandro Von Krannichfeldt et al. evaluated this paradigm for building energy systems in Combining Physics-based and Data-driven Modeling for Building Energy Systems. They demonstrated that residual models maintain predictive reliability during extreme, out-of-distribution environmental events where historical data patterns completely break down.

Why does this happen? In a standard black-box model, entering an unmapped region of the state space leads to erratic, unpredictable extrapolations. However, in a residual framework, such as the one proposed by Yuhan Liu et al. in Physics-Guided State-Space Model Augmentation Using Weighted Regularized Neural Networks, the neural network can be regularized to decay to zero when data is scarce. As a result, in the complete absence of reliable empirical data, the overall system gracefully falls back on the baseline physical laws.

In-Distribution:
Total Prediction = Physics Baseline + Neural Network Correction (High Accuracy)

Out-of-Distribution (Data-Sparse):
Total Prediction = Physics Baseline + 0 (Graceful Degradation to Reliable Physics)

This strategy translates effectively across diverse domains:

By treating machine learning as a corrective layer rather than the entire foundation, the system inherits the robust, universal generalization of classical physics. When the empirical data runs out, the laws of nature take over.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 208 papers
  2. Version 1 · 2026-10-04 · 433 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_physics_informed_generalization_physics__e17d20dd,
  title = {physics-informed generalization (physics-informed extrapolation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physics-informed-generalization-physics-informed-extrapolation/rdr_e17d20dd779146beb20b40ea09df516d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_physics_informed_generalization_physics__e17d20dd_v2,
  title = {physics-informed generalization (physics-informed extrapolation)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/physics-informed-generalization-physics-informed-extrapolation/rdr_e17d20dd779146beb20b40ea09df516d/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake