Abstract
Physics-guided Neural Networks (PGNNs) offer a promising path for modeling physical systems by integrating known laws into the learning process. However, these models often fail when multiple physical constraints "compete," leading the optimization into non-physical local minima. This paper introduces CoPhy-PGNN, a framework that employs adaptive weighting strategies—such as simulated annealing and "cold starting"—to balance these losses. Our results demonstrate that CoPhy-PGNN can solve complex eigenvalue problems with high accuracy and provide massive speedups over traditional numerical solvers, maintaining robust performance even in extrapolation scenarios.
Introduction
Deep learning is increasingly applied to scientific domains, yet standard "black-box" neural networks frequently struggle with data scarcity and poor generalization outside the training distribution. While PGNNs address this by penalizing violations of physical laws, many real-world problems—like solving eigenvalue equations in quantum mechanics or electromagnetics—involve competing physics losses. For example, a model might satisfy a characteristic equation but converge to an incorrect energy state. Simple additive loss functions often result in conflicting gradients, trapping models in non-generalizable local minima.
Methodology: The CoPhy-PGNN Framework
CoPhy-PGNN introduces a dynamic approach to balancing two specialized physics losses:
- Characteristic Loss (C-Loss): Ensures the predicted eigenvector $\hat{y}$ and eigenvalue $\hat{b}$ satisfy the fundamental equation $A\hat{y} = \hat{b}\hat{y}$.
- Spectrum Loss (S-Loss): Acts as a guide to push the network toward the specific eigenvalue of interest (e.g., the ground state).
The "secret sauce" of the framework is Adaptive Tuning. Rather than static weights, CoPhy-PGNN uses:
- Annealing $\lambda_S$: The Spectrum Loss starts with high influence to orient the model and gradually decays.
- Cold Starting $\lambda_C$: The Characteristic Loss is initially zero, allowing the model to locate the correct region of the solution space before refining the solution using a sigmoid-ramped physics constraint.
This strategy enables the use of unlabeled data, as physics losses do not require ground-truth labels to be calculated.
Key Insights and Loss Landscapes
A critical finding is that physics loss can be counterproductive if not managed correctly. A "Label-free" PGNN using only physics loss may perform worse than a black-box NN due to local minima.
As shown in the loss landscape visualizations, while many models find a minimum in training data loss, only CoPhy-PGNN successfully navigates to a solution that simultaneously minimizes test error and physics violations. The "Cold Start" strategy is essential here, preventing chaotic gradients from disrupting early training.
Experimental Results
The framework was evaluated on two domains: 4-particle Ising chains (Quantum Mechanics) and periodically stratified layers (Electromagnetic Propagation).
Accuracy and Extrapolation
In the quantum task, CoPhy-PGNN achieved a 99.50% Cosine Similarity with ground truth, outperforming standard PGNNs (97.97%) and black-box NNs (95.32%). Crucially, the model demonstrated superior extrapolation capabilities.
As shown in Figure 1, CoPhy-PGNN maintains near-perfect similarity even when extrapolating outside the training distribution, whereas other models' performance drops sharply when moving from ferromagnetic to paramagnetic phases.
Computational Efficiency
For large-scale electromagnetic problems ($401 \times 401$ matrices), CoPhy-PGNN provided predictions in just 0.043 seconds. In contrast, standard numerical solvers like NumPy required 93.7 seconds, representing a speedup of over three orders of magnitude.
Conclusion and Future Work
CoPhy-PGNN demonstrates that the timing and weighting of physical constraints are as important as the constraints themselves. By leveraging unlabeled data and adaptive loss scheduling, the framework provides a robust, high-speed alternative to traditional numerical methods. Future research will focus on automating hyper-parameter tuning for the annealing schedules and establishing stronger theoretical foundations for convergence in high-dimensional spaces.
Summary updated