When modeling complex physical systems—whether tracking a chaotic weather pattern, predicting the degradation of a hydrogen fuel cell, or controlling a robotic arm—we traditionally face a stark choice between two paradigms.
On one hand, first-principles models rely on centuries of scientific knowledge encoded as differential equations and conservation laws. They are highly interpretable and generalize well to unseen scenarios, but they are often oversimplified, computationally expensive, and struggle to capture messy, real-world irregularities.
On the other hand, data-driven models (such as deep neural networks) excel at extracting intricate patterns from massive datasets. However, they are notoriously "black-box" systems, requiring vast amounts of data to learn basic physical laws from scratch, and they frequently make physically impossible predictions when forced to extrapolate.
As Ulrich Rüde et al. note in their roadmap for computational science, reconciling these two paradigms is essential for creating robust, next-generation predictive frameworks (Research and Education in Computational Science and Engineering). The emerging research direction of Physics-Informed Machine Learning (PIML), or hybrid gray-box modeling, bridges this gap. By embedding physical laws directly into the structure and optimization of machine learning models, researchers can build systems that inherit the robustness of classical physics and the flexibility of deep learning.
The Intuition of Hybrid Modeling
To understand why hybrid modeling works, think of a physical system as a combination of a known macro-trend and an unknown micro-fluctuation.
A pure physics model captures the macro-trend (e.g., fluid dynamics or gravity) but misses the micro-fluctuations (e.g., non-linear friction, joint elasticity, or local environmental noise). A pure data-driven model tries to memorize both from scratch, treating fundamental conservation laws as arbitrary patterns to be learned.
In a comprehensive survey, Joe Watson and Oliver Weeger outline how embedding physical knowledge directly into machine learning architectures constrains the model's search space (Machine Learning with Physics Knowledge for Prediction: A Survey). Instead of searching through an infinite space of mathematically valid but physically absurd functions, the neural network is forced to look only at solutions that respect reality. This structural anchoring drastically improves data efficiency and ensures that predictions remain grounded in science.
Mechanics of Integration: Residuals and Constraints
How is this integration achieved in practice? Researchers generally employ three primary strategies to fuse data and physics:
1. Residual Learning and Model Augmentation
In this setup, a classical physics model handles the core, well-understood dynamics of the system. A neural network is then placed alongside it, tasked solely with predicting the error—the residual—between the idealized physical equations and real-world observations.
For example, Yong Tao et al. apply this to robotics, using a hybrid framework to compensate for torque errors that rigid mathematical equations fail to capture, thereby significantly boosting motion accuracy (Robot hybrid inverse dynamics model compensation method based on the BLL residual prediction algorithm).
2. Differentiable Modeling
Rather than keeping the physics engine and the neural network separate, differentiable modeling embeds physical equations directly into the neural network's computational graph. As Chaopeng Shen et al. explain, this allows gradients to flow seamlessly through both the physical and neural components (Differentiable modelling to unify machine learning and physical models for geosciences). The physical parameters themselves become learnable via standard gradient descent, unifying optimization and ensuring strict physical consistency.
3. Hardcoding Laws into Kernels and Architectures
Alternatively, physical constraints can be baked directly into the mathematical building blocks of the machine learning model. Maziar Raissi demonstrates this by encoding linear differential equations directly into Gaussian Process kernels, allowing the model to infer physical constants and hidden states from remarkably small, noisy datasets (Machine learning of linear differential equations using Gaussian processes).
Similarly, Nikolay Atanasov et al. enforce energy conservation and kinematic consistency by embedding Hamiltonian mechanics directly onto the geometric manifolds of Neural Ordinary Differential Equations (ODEs) (Hamiltonian-based Neural ODE Networks on the SE(3) Manifold For Dynamics Learning and Control).
Why Hybrid Models Extrapolate Better
The true test of any predictive model is extrapolation—predicting scenarios entirely outside the training data, such as a 100-year flood or an unprecedented mechanical failure. Purely data-driven models suffer from a "saturation ceiling" or make erratic predictions when encountering out-of-distribution data because they lack an underlying causal framework.
By anchoring the model to invariant physical laws, hybrid architectures maintain physical consistency even in unfamiliar territory. Eduardo Acuña Espinoza et al. demonstrated this when evaluating hybrid hydrological models for extreme flood forecasting (Analyzing the generalization capabilities of a hybrid hydrological model for extrapolation to extreme events). They found that while pure machine learning models failed to predict unprecedented peak discharges, the hybrid model successfully generalized. Because its underlying equations enforced mass balance constraints, the data-driven components were prevented from "cheating" or adjusting parameters in ways that violated physical reality.
Balancing Speed and Physical Consistency
Beyond accuracy, hybrid modeling solves a critical operational bottleneck: computational speed. Traditional numerical simulations (such as those used in climate modeling or fluid dynamics) are highly accurate but too slow for real-time applications or long-term forecasting. Conversely, pure deep learning models are fast but prone to physical drift over long horizons.
Fusing these paradigms yields massive computational savings without sacrificing physical validity. For example, Dmitrii Kochkov et al. introduced NeuralGCM, a hybrid model that integrates atmospheric physics with neural networks (Neural general circulation models for weather and climate). It achieves weather and climate forecast accuracy comparable to traditional general circulation models while offering orders-of-magnitude computational savings.
Similarly, Sibo Cheng et al. demonstrate that combining data assimilation with deep learning can drastically accelerate forecasting times while robustly handling sparse, moving sensor data (Efficient deep data assimilation with sparse observations and time-varying sensors).
By treating physics not as an alternative to machine learning, but as its foundational inductive bias, this research direction provides a scalable blueprint for modeling the complex, non-linear world around us.
Key Foundational Work
Gray-box identification of dynamic models for the bleaching operation in a pulp mill (2004) — contributed 2 concept elements to this direction.