Abstract
Machine learning models have been successfully used in many scientific and engineering fields. However, it remains difficult for a model to simultaneously utilize domain knowledge and experimental observation data. The application of knowledge-based symbolic AI represented by an expert system is limited by the expressive ability of the model, and data-driven connectionism AI represented by neural networks is prone to produce predictions that violate physical mechanisms. In order to fully integrate domain knowledge with observations, and make full use of the prior information and the strong fitting ability of neural networks, this study proposes theory-guided hard constraint projection (HCP). This model converts physical constraints, such as governing equations, into a form that is easy to handle through discretization, and then implements hard constraint optimization through projection. Based on rigorous mathematical proofs, theory-guided HCP can ensure that model predictions strictly conform to physical mechanisms in the constraint patch. The performance of the theory-guided HCP is verified by experiments based on the heterogeneous subsurface flow problem. Due to the application of hard constraints, compared with fully connected neural networks and soft constraint models, such as theory-guided neural networks and physics-informed neural networks, theory-guided HCP requires fewer data, and achieves higher prediction accuracy and stronger robustness to noisy observations.
Introduction
While deep learning has revolutionized many fields, its application in "hard" sciences like physics, fluid dynamics, and hydrology faces a significant hurdle: standard neural networks are essentially "black boxes" that lack an inherent understanding of the laws of nature. Consequently, they can produce predictions that violate fundamental principles such as the conservation of mass or energy.
Existing solutions, such as Physics-Informed Neural Networks (PINNs), typically utilize soft constraints. These models add penalty terms to the loss function to "encourage" physical compliance. However, this remains a suggestion rather than a requirement; models can still deviate from physical laws, particularly when training data is noisy. This paper introduces Theory-guided Hard Constraint Projection (HCP), a method that moves beyond suggesting physical laws to strictly enforcing them through a mathematical projection layer.
Methodology: The HCP Framework
The core innovation is the HCP layer, which functions as a non-trainable "physical filter" at the end of the neural network. Unlike soft-constraint models that rely solely on loss function penalties, HCP ensures exact physical compliance within local patches.
Theoretical Foundation
The theory-guided framework typically combines data-driven observation loss with domain-specific losses derived from governing equations and boundary conditions.
The HCP method builds upon this by implementing a three-step projection process:
- Equation Discretization: Continuous governing equations (e.g., subsurface flow PDEs) are converted into discrete algebraic forms using finite difference schemes.
- Matrix Decomposition: The discretized equations are partitioned into a constraint matrix ($A$) representing the physical mechanism and a prediction matrix ($H$) containing the network's raw outputs.
- Hard Constraint Projection: The model utilizes a projection matrix $P = I - A^T(AA^T)^{-1}A$ to map predictions onto a hyperplane where physical equations are perfectly satisfied ($AH = 0$).
To handle boundary conditions, the authors employ "ghost cells"—virtual cells that enforce boundary logic—ensuring the model respects physical limits like constant pressure or no-flow zones.
Experimental Evaluation and Results
The method was evaluated using a heterogeneous subsurface flow problem, with synthetic data generated via MODFLOW. The model was trained on sparse observations (the first 18 time steps) and tasked with predicting the subsequent 32 steps.
Key Findings
- Data Efficiency: HCP significantly outperformed standard ANNs and soft-constrained models (TgNN) when collocation points were scarce. With only 40 points, HCP maintained high accuracy where others failed.
- Robustness to Noise: In tests with up to 40% noise, HCP remained stable. Purely data-driven models, by contrast, were highly sensitive to such perturbations.
- Extrapolation and Speed: HCP accurately predicted future states outside its training range, a common failure point for standard NNs. Furthermore, inference was 32 to 445 times faster than traditional numerical simulations.
- The "River" Analogy: Optimization in HCP is akin to forcing every step of a descent to stay exactly within a "river" (the physical law), rather than just staying near it as in soft-constraint models. Interestingly, the HCP layer often only needs to adjust raw outputs by less than 1% to achieve this, providing a much clearer optimization direction.
Conclusion
Theory-guided HCP offers a robust bridge between data-driven neural networks and knowledge-based physical modeling. While it currently focuses on local constraint patches, future work will address global compliance and sensitivity to extreme outliers. By enforcing physical laws as hard constraints, HCP provides a reliable and efficient tool for scientific machine learning applications.
Summary updated