Deep learning excels at pattern recognition but frequently falters when data is sparse or when predicting outcomes outside the training distribution. In physical systems, pure black-box models often yield physically impossible predictions because they lack an inherent understanding of universal constraints, such as the conservation of energy or mass. Conversely, traditional mechanistic models are grounded in first principles but can be computationally prohibitive or incomplete due to unmodeled real-world complexities.
As highlighted by Alfred O. Hero III et al. in their panel discussion summary, a major paradigm shift is underway: integrating rigorous physical models with data-driven algorithms to achieve interpretability, efficiency, and robustness. This hybrid approach does not replace classical physics; instead, it uses domain knowledge to guide, constrain, and accelerate neural networks.
The Hybridization Spectrum
The integration of physics and machine learning is not a one-size-fits-all strategy. As Marcus Haywood-Alexander et al. argue in their survey, this integration exists on a spectrum determined by our confidence in the underlying physical laws.
When the physical laws governing a system are perfectly understood, data is best used to calibrate or update model parameters. However, when there is high physical uncertainty or unmodeled environmental noise, neural networks can be deployed to learn residuals or act as flexible function approximators bounded by physical constraints.
Pure Physics Model <---------------------------------------------> Pure Data-Driven Model
(Rigid, Interpretable) Parameter Tuning Residual Learning (Flexible, Black-Box)
Mechanism 1: Penalizing the Loss Function (Soft Constraints)
The most common way to enforce physical laws is through soft constraints in the objective function, a method exemplified by Physics-Informed Neural Networks (PINNs). In a comprehensive review of plasma science, Richard Archibald et al. explain how known physical laws are embedded directly into the loss function as regularization terms [2022 Review of Data-Driven Plasma Science].
In this setup, the total loss function is split into two parts: $$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{data}} + \lambda \mathcal{L}_{\text{physics}}$$
Where $\mathcal{L}_{\text{data}}$ measures standard empirical error (e.g., mean squared error against labeled points), and $\mathcal{L}_{\text{physics}}$ evaluates how well the network's outputs satisfy a given differential or thermodynamic equation. If a network predicting fluid flow outputs a velocity field that violates mass conservation, $\mathcal{L}_{\text{physics}}$ penalizes the model heavily. This reshapes the optimization landscape, forcing the gradient descent process to favor physically plausible solutions even in regions where training data is non-existent.
Mechanism 2: Hard Architectural Constraints and Differentiable Physics
While loss-based penalties act as strong guidelines, they are "soft" constraints that a network can still bypass if the optimization gets stuck in a local minimum. A more rigid approach embeds physics directly into the network architecture itself, making violations mathematically impossible.
One way to achieve this is through differentiable physical decoders. Nikita Basargin et al. developed a hybrid framework for soil moisture estimation where a neural network acts as an encoder, but the reconstruction step is handled by a differentiable physical model [Explainable Physical PolSAR Autoencoders for Soil Moisture Estimation]. Because the backpropagation path passes through exact electromagnetic equations, the latent representations are forced to remain physically meaningful.
Similarly, in medical imaging, Chen Qin et al. distilled complex biomechanical laws into a learnable latent space to constrain myocardial motion tracking [Biomechanics-informed Neural Networks for Myocardial Motion Tracking in MRI]. By restricting the network's output space to a manifold of physiologically realistic deformations, the model generalizes to unseen patient data without memorizing superficial image patterns.
Mechanism 3: Residual and Grey-Box Modeling
In many engineering applications, we possess a solid but incomplete physical model. Rather than discarding this model and starting from scratch with a neural network, "grey-box" modeling uses machine learning to capture only the unexplained variance or residuals.
Manu Lahariya et al. demonstrate this concept in industrial cooling systems, combining the predictive flexibility of deep learning with the structural constraints of thermodynamics [Physics-Informed LSTM Network for Flexibility Identification in Evaporative Cooling System]. The physical model handles the macro-level energy balance, while an LSTM network models the highly non-linear, transient fluctuations that analytical equations fail to capture.
This strategy is also highly effective for scientific discovery. As noted by David Humphreys et al., using data-driven models to isolate the discrepancies between first-principles models and experimental observations allows physicists to identify missing terms in theoretical equations, bridging the gap between theory and reality [Advancing Fusion with Machine Learning Research Needs Workshop Report].
Why It Works: Restricting the Search Space
To understand why physics-informed machine learning outperforms pure deep learning, it helps to look at the geometry of the model's search space. Unconstrained neural networks possess an immense capacity to fit arbitrary functions. While this flexibility is useful for vision or language, it makes them prone to learning spurious correlations or systematic biases when applied to physical systems.
As Neil D. Lawrence et al. point out, replacing unconstrained networks with causally-informed versions eliminates these systematic biases, which is essential for reliable scientific forecasting [Machine Learning for Science: Mathematics at the Interface of Data-driven and Mechanistic Modelling]. By restricting the network's hypotheses to a manifold governed by universal laws, the model requires significantly less data to train, resists overfitting to noisy inputs, and generalizes safely when extrapolating to unseen environments [Olga Fink et al.].
[ Unconstrained Search Space (All possible mathematical functions) ]
|
v
[ Physics-Constrained Manifold (Only functions conserving mass/energy) ]
|
v
[ Optimized Solution (Accurate, robust, and physically sound) ]
Furthermore, grounding neural networks in geometric numerical integration ensures that long-term simulations preserve fundamental physical properties like energy or symplecticity. As discussed by Elena Celledoni et al., this structural grounding prevents the numerical drift and compounding errors that plague standard deep learning rollouts over long time horizons [Preface].
Accelerating Rather Than Replacing
Hybrid modeling can also be used to accelerate specific bottlenecks within a physical simulation without altering the simulation's core equations. For example, Thomas Frerix demonstrates that using a neural network to learn an inverse observation operator can vastly speed up the search for initial conditions in variational data assimilation [Variational Data Assimilation with a Learned Inverse Observation Operator]. Because the neural network only accelerates the initialization phase while leaving the exact physical forward solver intact, the resulting forecasts remain perfectly consistent with physical laws.