Theory-guided hard constraint projection (HCP): A knowledge-based data-driven scientific machine learning method

Yuntian Chen, Dou Huang, Dongxiao Zhang, Junsheng Zeng, Nanzhe Wang, Haoran Zhang, Jinyue Yan

This paper introduces a novel approach to Scientific Machine Learning (SciML) that moves beyond "suggesting" physical laws to a model and starts "enforcing" them. By implementing a hard constraint projection layer, the authors demonstrate how to make neural networks more reliable, data-efficient, and physically consistent.

TL;DR

The authors propose Theory-guided Hard Constraint Projection (HCP), a method that forces neural network predictions to strictly satisfy governing physical equations (like PDEs) through a mathematical projection layer. Unlike standard physics-informed models that use "soft" penalty terms, HCP ensures exact physical compliance within local patches, significantly improving accuracy and robustness—especially when data is sparse or noisy.

What Problem Does This Paper Address?

While deep learning has revolutionized many fields, its application in "hard" sciences (physics, fluid dynamics, hydrology) faces a major hurdle: standard neural networks are "black boxes" that don't understand the laws of nature. They can easily produce predictions that violate fundamental principles like the conservation of mass or energy.

Existing solutions, such as Physics-Informed Neural Networks (PINNs), typically use soft constraints. They add a penalty term to the loss function that "encourages" the model to follow the physics. However, this is just a suggestion; the model can still deviate from the physics at specific points, especially when dealing with noisy experimental data. The challenge is to create a model that has the fitting power of a neural network but the "common sense" and stability of a traditional physical solver.

Methods

The core of the paper is the HCP layer, which acts as a non-trainable "physical filter" at the end of the network. The process follows three main steps:

  1. Equation Discretization: The continuous governing equations (e.g., subsurface flow PDEs) are converted into a discrete algebraic form using finite difference schemes.
  2. Matrix Decomposition: The discretized equation is split into a constraint matrix ($A$) representing the physical mechanism and a prediction matrix ($H$) containing the neural network's raw outputs.
  3. Hard Constraint Projection: The model uses a projection matrix $P = I - A^T(AA^T)^{-1}A$ to map the network's predictions onto a "hyperplane" where the physical equations are perfectly satisfied ($AH = 0$).

To handle boundary conditions (like constant pressure or no-flow zones), the authors adapt the concept of "ghost cells" from computational fluid dynamics, essentially "padding" the simulation area with virtual cells that enforce boundary logic.

Datasets

The method was evaluated using a heterogeneous subsurface flow problem:

  • Source: Synthetic data generated via MODFLOW, a gold-standard software for groundwater modeling.
  • Setup: A 2D transient saturated flow in a 51x51 grid over 50 time steps.
  • Challenge: The model was trained on only the first 18 time steps (sparse observation) and tasked with predicting the remaining 32 steps (extrapolation).

Key Insights and Ideas

  • The "River" Analogy: The authors compare optimization to climbing down a mountain. A data-driven model just goes down the steepest slope. A soft-constraint model tries to stay near a "river" (the physical law) but might drift. HCP forces every single "foothold" to be exactly in the river.
  • Small Adjustments, Big Impact: The paper reveals that the HCP layer often only needs to adjust the raw neural network output by less than 1% to satisfy the physics. However, this tiny correction provides a much clearer "optimization direction," allowing the model to converge faster and more accurately.
  • Local to Global: By applying constraints to small "patches" rather than the entire field at once, the method remains computationally efficient while still capturing the global physical behavior.

Key Results

  • Superior Data Efficiency: HCP outperformed both standard ANNs and soft-constrained models (TgNN) when collocation points were scarce. With only 40 points, HCP's error was significantly lower than its competitors.
  • Robustness to Noise: In tests with up to 40% noise in observations, HCP maintained stable and accurate predictions, whereas purely data-driven models failed.
  • Extrapolation Power: Because it is anchored in physical laws, HCP could accurately predict future states of the flow field that were entirely outside the range of its training data—a task where standard ANNs typically fail.
  • Inference Speed: Once trained, the HCP model provided predictions 32 to 445 times faster than traditional numerical simulations.

Limitations and Future Work

  • Global vs. Local Compliance: While HCP guarantees physics are satisfied within the local "constraint patches," it does not mathematically guarantee that the entire global field is perfectly satisfied simultaneously (though it comes very close).
  • Sensitivity to Outliers: The authors note that a single extreme outlier in a constraint patch can "pull" the projection of other points in that patch in the wrong direction. Future work may involve integrating outlier detection before the projection step.
  • Computational Scaling: Larger constraint patches lead to higher computational costs. Finding the optimal balance between patch size and efficiency is an ongoing area of research.

Summary updated

Abstract

Machine learning models have been successfully used in many scientific and engineering fields. However, it remains difficult for a model to simultaneously utilize domain knowledge and experimental observation data. The application of knowledge-based symbolic AI represented by an expert system is limited by the expressive ability of the model, and data-driven connectionism AI represented by neural networks is prone to produce predictions that violate physical mechanisms. In order to fully integrate domain knowledge with observations, and make full use of the prior information and the strong fitting ability of neural networks, this study proposes theory-guided hard constraint projection (HCP). This model converts physical constraints, such as governing equations, into a form that is easy to handle through discretization, and then implements hard constraint optimization through projection. Based on rigorous mathematical proofs, theory-guided HCP can ensure that model predictions strictly conform to physical mechanisms in the constraint patch. The performance of the theory-guided HCP is verified by experiments based on the heterogeneous subsurface flow problem. Due to the application of hard constraints, compared with fully connected neural networks and soft constraint models, such as theory-guided neural networks and physics-informed neural networks, theory-guided HCP requires fewer data, and achieves higher prediction accuracy and stronger robustness to noisy observations.
BibTeX
@article{chen2021theory,
  author = {Yuntian Chen and Dou Huang and Dongxiao Zhang and Junsheng Zeng and Nanzhe Wang and Haoran Zhang and Jinyue Yan},
  title = {{Theory-guided hard constraint projection (HCP): A knowledge-based data-driven scientific machine learning method}},
  journal = {Journal of Computational Physics},
  year = {2021},
  url = {https://openreview.net/forum?id=atYo3Wnu9PL},
}

Loading related content...

Figure