When applying deep learning to physical systems, standard neural networks face a fundamental problem: they do not know the laws of physics. A vanilla network is perfectly content to violate the conservation of mass, allow energy to drift infinitely, or let fluids leak through solid, impermeable walls.
To fix this, early frameworks for physics-informed machine learning relied on soft constraints. They added physical laws—expressed as partial differential equations (PDEs)—directly into the loss function as penalty terms. The total loss became a balancing act:
$$\mathcal{L}_{\text{total}} = \mathcal{L}_{\text{data}} + \lambda \mathcal{L}_{\text{physics}}$$
While intuitive, this approach introduces a destructive optimization tug-of-war. Because the optimizer treats physics as a suggestion rather than a strict rule, it often minimizes the overall loss by allowing minor physical violations to achieve better data fitting. As Lu et al. observe, this soft enforcement allows models to produce solutions that are mathematically optimized but physically impossible to realize.
To build truly reliable models for scientific and engineering applications, a growing body of research shifts the paradigm from penalizing physical violations to architecturally preventing them. By baking physical laws directly into the network's structure, these models satisfy constraints to machine precision by design.
The Failure Modes of Soft Constraints
Relying on soft penalties introduces several critical vulnerabilities that compromise model integrity:
- Accumulated Drift and Instability: In long-term simulations, even microscopic violations of conservation laws accumulate over time. Lee et al. point out that soft penalties frequently result in numerical instability or non-physical behavior like energy drift during long-term forecasting.
- Boundary Violations: When boundary conditions are enforced via loss penalties, the model often fails to strictly adhere to them. As Saad et al. demonstrate, this allows unphysical behaviors like fluid leaking through solid boundaries or heat escaping through perfect insulators.
- Unphysical Hallucinations: In regimes where data is sparse or noisy, soft-constrained models tend to generate visually plausible fields that flagrantly violate underlying governing equations (Li et al.).
- Hyperparameter Sensitivity: Balancing competing loss terms requires tedious, problem-specific hyperparameter tuning of the penalty weights $\lambda$. Gao et al. note that this balancing act often fails to strictly satisfy essential constraints, leading to physically inconsistent results in both forward and inverse problems.
Shifting the Hypothesis Space
The core intuition behind hard-constrained architectures is a geometric one: shrink the hypothesis space.
Instead of allowing a neural network to search the entire space of mathematical functions and punishing it when it leaves the physically valid manifold, we alter the network architecture so that its output space is the physical manifold.
Soft Constraint (Penalty-based):
[ Unconstrained Search Space ] ---> Optimizer tries to pull model toward ---> [ Physical Manifold ]
(Model can still drift out)
Hard Constraint (By Construction):
[ Network Architecture ] ---> Outputs are mathematically restricted to ---> [ Physical Manifold ]
(Model cannot physically leave)
By restricting the hypothesis space before optimization begins, it becomes mathematically impossible for the network to violate the laws of physics, regardless of how noisy the training data is or how poorly the optimizer performs.
Mechanisms for Hard-Coding Physics
Researchers have developed three primary architectural strategies to enforce hard physical constraints: exact functional spaces, differentiable projection layers, and structure-preserving dynamical formalisms.
1. Exact Functional Spaces and Priors
One approach is to restrict the network's output to a specific mathematical space where the governing equations are satisfied automatically.
For conservation laws (like mass or charge conservation), Richter-Powell et al. leverage the mathematics of differential forms to ensure that network predictions are exactly divergence-free by design, satisfying the continuity equation without any penalty terms. Similarly, Ghosh et al. utilize holomorphic functions and curl-driven fields to bake Laplace's equation directly into the network architecture.
For boundary conditions, Saad et al. perform "structural surgery" on the operator's kernel to mathematically guarantee that boundary constraints are satisfied exactly. Outside of deep learning, Lange-Hegermann uses symbolic algebra (Gröbner bases) to construct multi-output Gaussian Process priors whose realizations strictly satisfy linear partial differential equations and boundary conditions without numerical approximations.
2. Differentiable Projection Layers and Solvers
When constraints cannot be easily baked into the functional form, networks can pass their unconstrained intermediate outputs through an explicit mathematical projection layer.
Chen et al. introduced a hard constraint projection (HCP) layer that maps the network's predictions back onto the valid physical subspace. For network-based engineering systems, Dogoulis et al. implement a Moore-Penrose pseudoinverse projection layer to enforce Kirchhoff’s Current Law exactly, ensuring zero physical violations even in highly volatile contingency scenarios.
Alternatively, a differentiable physical solver can be embedded directly as a layer within the network. By utilizing the implicit function theorem, Négiar et al. develop a PDE-Constrained Layer (PDE-CL) that forces the model to satisfy physical constraints as hard requirements, achieving significantly higher accuracy and faster convergence than traditional soft-constrained models.
3. Structure-Preserving Dynamical Formalisms
For time-evolving systems, physics can be hard-coded by embedding classical mechanics frameworks directly into the state transition equations.
To prevent energy drift in conservative systems, the authors embed Hamiltonian formalisms into neural ordinary differential equations (Lee et al.). However, real-world systems are often dissipative and irreversible. To handle thermodynamics, Lee et al. embed metriplectic algebraic structures (the GENERIC formalism) into the network architecture, guaranteeing both energy conservation and entropy growth by construction.
Similarly, Zakwan et al. leverage Irreversible port-Hamiltonian Systems (IPHS) to guarantee thermodynamic consistency by design, outperforming black-box models in long-term multi-physics simulations. When the system operates on complex geometric manifolds, Duruisseaux et al. enforce the geometric constraints of Lie groups to prevent the model from generating illegal physical states during inference.
Why Hard Constraints Generalize Better
The benefits of hard constraints extend far beyond physical correctness; they fundamentally change how the model learns.
When training data is scarce, noisy, or out-of-distribution, soft-constrained models easily collapse because the data loss term becomes uninformative or misleading (Bahmani). In contrast, hard-constrained architectures remain bounded within the physically valid hypothesis space.
As demonstrated by Harder et al. in climate downscaling applications, embedding these hard physical constraints directly within the network architecture ensures 100% physical consistency while simultaneously improving data efficiency and predictive accuracy. By removing the burden of learning the laws of physics from scratch, the network can dedicate its entire capacity to learning the complex, unmodeled residual dynamics of the data.