Deep learning has revolutionized our ability to model complex, high-dimensional systems. In fields ranging from meteorology to fluid dynamics, neural networks can act as surrogate models or emulators, producing predictions orders of magnitude faster than traditional numerical solvers.
However, standard deep learning models suffer from a fundamental flaw when applied to the physical world: they are "physics-blind." A standard neural network treats physical states merely as arrays of numbers, optimizing for statistical correlations rather than physical laws. Because these models lack an inherent understanding of principles like the conservation of mass, energy, or momentum, they frequently produce predictions that are statistically plausible but physically impossible.
To bridge this "physical consistency gap," researchers are developing frameworks that embed physical laws directly into data-driven architectures. Understanding why this gap occurs—and how injecting physical priors resolves it—is essential for building reliable machine learning models for scientific discovery.
The Problem: Numerical Drift and Unphysical Shortcuts
When a standard neural network is trained to predict the next state of a dynamical system, it minimizes a purely statistical loss function, such as Mean Squared Error (MSE). While this approach works well for short-term predictions, it fails catastrophically over long horizons due to two interrelated phenomena: numerical drift and unphysical shortcuts.
Numerical Drift
In a long-term simulation, a model must feed its own predictions back into itself step after step. If a neural network has a tiny, unconstrained error that violates mass conservation by just 0.1% at step one, that error compounds over thousands of timesteps.
As Brian Henn et al. observe in their work on atmospheric modeling, ACE2: accurately learning subseasonal to decadal atmospheric variability and forced responses, this lack of inherent physical constraints causes errors in mass or moisture to accumulate over time. This numerical drift eventually drives the simulation into highly unrealistic or unstable states. Similarly, Kim Stachenfeld et al. demonstrate in Learned Simulators for Turbulence that without inductive biases to strictly satisfy conservation laws, data-driven fluid simulators exhibit physical drift where conserved quantities gradually deviate from their true values over extended periods.
Unconstrained Model:
[Initial State] -> [Prediction + Small Mass Error] -> [Input Next Step] -> ... -> [Catastrophic Drift]
Constrained Model:
[Initial State] -> [Prediction (Mass Conserved)] -> [Input Next Step] -> ... -> [Stable Trajectory]
Unphysical Shortcuts
Because statistical loss functions treat all errors of the same magnitude equally, a network will happily minimize loss by taking mathematically valid but physically nonsensical shortcuts. For instance, in a precipitation or flood model, an unconstrained network might minimize error by simply "creating" water out of thin air or allowing fluid to flow uphill if it matches a statistical pattern in the training data.
As Zhihan Gao et al. explain in PreDiff: Precipitation Nowcasting with Latent Diffusion Models, purely data-driven models often generate predictions that appear visually realistic but are meteorologically impossible because they violate fundamental thermodynamic or physical constraints. In closed-loop control applications, Connor Schenck demonstrates in Learning Robotic Manipulation of Granular Media that models lacking explicit physical priors (like mass conservation) learn non-physical shortcuts that satisfy training objectives but fail completely during multi-step planning.
Why It Works: Constraining the Hypothesis Space
The core intuition behind physics-constrained machine learning is simple: we shrink the space of functions the neural network can learn.
An unconstrained neural network searches an incredibly vast hypothesis space, which includes millions of functions that fit the training data perfectly but violate the laws of physics. When the model encounters noisy data or out-of-distribution scenarios, it easily strays into these unphysical regions. By enforcing physical consistency, we erect mathematical guardrails that restrict the network's search space only to functions that obey physical laws.
Hypothesis Space of All Possible Functions
+-------------------------------------------------------+
| Unconstrained Network Search Area |
| (Includes physically impossible functions) |
| |
| +-----------------------------------------------+ |
| | Physically Consistent Manifold | |
| | (Mass, Energy, Momentum Conserved) | |
| | | |
| | * Constrained Network Search Area | |
| +-----------------------------------------------+ |
+-------------------------------------------------------+
This restriction provides three massive advantages:
- Data Efficiency: The model doesn't need to waste parameters or training data "re-learning" that mass cannot be destroyed; that rule is pre-programmed.
- Robustness to Noise: If the training data is corrupted, the physical constraints prevent the model from fitting to the noise if that noise implies an unphysical state.
- Stable Extrapolation: When predicting scenarios outside the training data (such as long-term climate projections), the model remains anchored to physical reality.
Mechanisms for Enforcing Physical Consistency
Researchers employ three primary strategies to embed physical knowledge into machine learning models: soft constraints, hard architectural constraints, and hybrid/surrogate coupling.
1. Soft Constraints via Loss Regularization
The most straightforward way to enforce physics is to add a penalty term to the loss function that represents a physical law or a partial differential equation (PDE).
$$\text{Loss} = \text{Loss}_{\text{data}} + \lambda \text{Loss}_{\text{physics}}$$
For example, Paris Perdikaris et al. utilize this in Adversarial uncertainty quantification in physics-informed neural networks, treating governing PDEs as a regularization mechanism to ensure deep generative models remain physically consistent even when trained on small, noisy datasets. Similarly, Marco Lorenzi in Constraining the Dynamics of Deep Probabilistic Models embeds structural and differential constraints directly into Deep Gaussian Processes. While soft constraints are easy to implement, they act like "rumble strips" on a highway—they penalize deviations but do not completely prevent the model from crossing physical boundaries if the data loss is strong enough.
2. Hard Constraints via Architectural Design
To guarantee physical consistency, researchers design specialized network architectures where physical laws are mathematically impossible to violate.
A prime example is embedding conservation laws directly into the network's mathematical operations. In Machine Learning That Reproduces Physical Phenomena from Data, Yusuke Tanaka embeds the law of energy conservation directly into a model's architecture using Hamiltonian mechanics, achieving highly accurate simulations even with sparse or corrupted training data. In chemistry, Joonyoung F. Joung et al. introduce this concept to molecular modeling in Electron flow matching for generative reaction mechanism prediction, strictly enforcing mass conservation so that atoms or electrons cannot magically appear or disappear during predicted chemical reactions.
3. Hybrid Modeling and Latent Coupling
Another powerful approach is to "sandwich" data-driven components together with traditional physical solvers or to ground learned latent spaces in physical variables.
Instead of letting a neural network predict an entire physical field from scratch, a hybrid model might use a traditional numerical solver to handle the coarse, well-understood physics, while using a neural network to handle the complex, unresolved scales. For instance, Ali Can Bekar et al. develop Hybrid Latent Representations for PDE Emulation, which augments low-resolution physical fields with spatially structured latent channels. This anchors the learned representation to known physical laws and prevents the model from drifting over long simulation horizons.
In weather forecasting, Peiyuan Liu et al. propose a similar philosophy in Mitigating Time Discretization Challenges with WeatherODE: A Sandwich Physics-Driven Neural ODE for Weather Forecasting, blending wave equation theory with a hybrid CNN-ViT architecture to ensure continuous-time predictions remain physically bounded. In data assimilation, Sibo Cheng et al. demonstrate in Ensemble latent assimilation with deep learning surrogate model: application to drop interaction in a microfluidics device how an ensemble-based assimilation scheme can continuously correct an LSTM's latent space drift using real-time physical observations.
Conclusion
Purely data-driven models are exceptionally good at interpolation, but they lack the structural discipline required to simulate the physical universe over time. By forcing neural networks to respect conservation laws—whether through loss regularization, hard architectural constraints, or hybrid coupling—we transform them from black-box statistical pattern matchers into reliable, physically consistent emulators capable of accelerating scientific discovery without losing touch with reality.