Standard deep learning models excel at finding patterns in data, but they are notoriously blind to the laws of nature. When applied to physical systems—such as fluid dynamics, climate modeling, or robotics—a vanilla neural network can easily predict physically impossible states, such as creating mass out of nothing or allowing energy to drift infinitely over time.
To fix this, early approaches in physics-informed machine learning introduced "soft constraints." By adding physical equations as penalty terms to the loss function, the optimizer is discouraged from violating physical laws. However, as Chuizheng Meng et al. note in their survey, When physics meets machine learning: a survey of physics-informed machine learning, this creates a difficult multi-task optimization problem. The network enters a "tug-of-war" between minimizing statistical data error and satisfying physical penalties, often settling for a compromise that still violates fundamental conservation laws.
A powerful alternative paradigm has emerged: hard architectural constraints. Instead of punishing a network for breaking the laws of physics, researchers are redesigning neural network architectures so that violating physical laws is mathematically impossible. By restricting the network's hypothesis space to a physically valid manifold, these models achieve machine-precision consistency, require significantly less training data, and generalize robustly to unseen scenarios.
The Subspace Projection Approach
The most direct way to enforce a hard constraint is to let a neural network generate an unconstrained prediction, and then immediately project that prediction onto a subspace of physically valid solutions before the output is finalized.
[ Input ] ---> ( Neural Network Layers ) ---> [ Unconstrained Output ]
|
v
[ Physically Valid Prediction ] <--- [ Non-Trainable Projection Layer ]
Yuntian Chen et al. formalized this in Theory-guided hard constraint projection (HCP): a knowledge-based data-driven scientific machine learning method. They introduced a non-trainable projection layer at the end of the network that maps raw outputs directly onto a subspace defined by discretized physical equations. Because this mapping happens during both training and inference, the network is structurally incapable of producing an invalid state.
This projection strategy is highly effective for enforcing conservation laws. For instance, Tom Beucler et al. developed Architecture-Constrained networks (ACnets) in Enforcing Analytic Constraints in Neural Networks Emulating Physical Systems, which hard-code mass and energy balances into climate emulators. Similarly, Paula Harder et al. demonstrated in Hard-Constrained Deep Learning for Climate Downscaling that integrating a specialized renormalization layer forces high-resolution climate outputs to perfectly match low-resolution mass averages, eliminating the unphysical "hallucinations" common in standard super-resolution networks.
Physics by Construction: Mathematical Parameterization
Rather than projecting an invalid output back into a valid space, a more elegant method is to parameterize the network's internal operations so that its outputs are physically valid by construction.
Consider the problem of modeling incompressible fluid flow, where the velocity field must be divergence-free ($\nabla \cdot \mathbf{u} = 0$). Instead of predicting the velocity components directly and trying to force them to be divergence-free, Peirong Liu et al. designed a framework called YETI in Discovering Hidden Physics Behind Transport Dynamics. YETI forces the neural network to predict a vector potential ($\mathbf{A}$) rather than the velocity field itself. The actual velocity is then computed as the curl of this potential ($\mathbf{u} = \nabla \times \mathbf{A}$). Because the divergence of a curl is mathematically always zero ($\nabla \cdot (\nabla \times \mathbf{A}) = 0$), the network's predicted velocity field is guaranteed to be perfectly incompressible, regardless of the weights or training state.
This concept of leveraging mathematical identities spans several domains:
- Conservation Laws via Differential Forms: Ning Liu et al. developed clawNOs in Harnessing the Power of Neural Operators with Automatically Encoded Conservation Laws, utilizing the calculus of differential forms to ensure that conservation laws are structurally satisfied across complex fluid simulations.
- Positive Definiteness: In structural mechanics, material stiffness tensors must be positive definite to be physically realistic. Jonathan Stollberg achieved this in Multiscale topology optimization of functionally graded lattice structures based on physics-augmented neural network material models by forcing the network to output the components of a Cholesky decomposition matrix, which is then multiplied by its transpose to construct a guaranteed positive definite tensor.
Embedding Differentiable Solvers and Structural Masks
For systems governed by complex partial differential equations (PDEs) or strict hardware boundaries, hard constraints can be enforced by embedding traditional mathematical operations directly into the network layers.
Geoffrey Négiar et al. pioneered this in Learning differentiable solvers for systems with hard constraints by introducing a PDE-Constrained Layer (PDE-CL). By leveraging the implicit function theorem, they embedded a differentiable numerical solver directly into the network architecture. The network learns to predict parameters for the solver, while the solver layer ensures that the final output satisfies the governing physical equations to within strict numerical tolerances.
In other applications, physical boundaries dictate how information can flow through space and time. For example, in traffic flow, information cannot travel faster than the physical speed of the vehicles or backward against the flow of a one-way street. In Incorporating Kinematic Wave Theory into a Deep Learning Method for High-Resolution Traffic Speed Estimation, Bilal Thonnam Thodi et al. enforced these physical boundaries by structurally masking out kernel regions in their network. This architectural restriction prevents the model from learning non-physical spatiotemporal correlations, while simultaneously reducing the number of trainable parameters.
Why It Works: The Optimization Advantage
Shifting physical laws from the loss function to the architecture fundamentally alters the optimization landscape. When constraints are handled via soft loss penalties, the optimizer must navigate a highly non-convex landscape with competing objectives, frequently getting stuck in local minima that yield unphysical solutions.
By contrast, hard architectural constraints restrict the optimizer's search space exclusively to valid solutions. As Muhammad Zakwan et al. demonstrated with their PC-NODEs framework in Physically Consistent Neural ODEs for Learning Multi-Physics Systems*, embedding thermodynamics directly into the network structure guarantees energy conservation and entropy production by design. This structural guarantee prevents the model from drifting into chaotic, unphysical states during long-term simulations.
Ultimately, as Davide Murari argues in Hamiltonian Matching for Symplectic Neural Integrators, making unphysical behavior mathematically impossible provides a level of robustness and reliability that training-time regularizers simply cannot match. Furthermore, Hao Chen et al. showed in Physics-Informed Neural Networks with Hard Linear Equality Constraints that replacing soft penalties with hard architectural constraints prevents the accumulation and propagation of small errors across interconnected systems. By removing the burden of learning basic physical invariants from the data, the network can focus its capacity entirely on capturing the complex, non-linear dynamics of the target system.
Key Foundational Work
Learning Audio Feedback for Estimating Amount and Flow of Granular Material (2018) — contributed 2 concept elements to this direction.