Introduction
Modeling the dynamics of physical systems, such as planetary orbits or complex mechanical linkages, traditionally requires either deep domain expertise or extensive observational data. While standard neural networks can approximate these dynamics, they often struggle to generalize because they lack an inherent understanding of physical laws, such as the conservation of energy.
Recent advancements like Hamiltonian Neural Networks (HNNs) and Lagrangian Neural Networks (LNNs) address this by embedding physical principles directly into the model architecture. However, these models typically learn the derivatives of the system's energy. This focus on gradients leaves the absolute energy level—the integration constant—arbitrary. In "small data" regimes, where observations are sparse, this lack of a global energy reference leads to poor generalization and numerical instability. This paper introduces a simple yet effective energy-level regularization technique to anchor these models, significantly improving performance when training data is scarce.
Methodology: Energy-Level Regularization
The authors propose a novel regularization term to force the network to respect not only the gradients of energy (the equations of motion) but also the absolute energy levels of the system.
Standard HNNs minimize a loss function based on the difference between predicted and observed time derivatives of the system's state (position and momentum). The proposed innovation adds a new term to the loss function: $$\mathcal{L}_{reg} = \lambda_H (H_\phi(z^{(i)}) - \hat{H}_i)^2$$ where $H_\phi(z^{(i)})$ is the Hamiltonian (total energy) predicted by the network, $\hat{H}_i$ is the actual total energy at that state, and $\lambda_H$ is a hyperparameter controlling the regularization strength.
By penalizing deviations from the true total energy, the model receives a stronger inductive bias. This "anchors" the Hamiltonian, resulting in a more accurate energy landscape even with minimal data points. This approach was applied to both standard HNNs and Constrained HNNs (CHHNs).
Experimental Evaluation
The method was evaluated on two physical benchmarks: a single pendulum (1 degree-of-freedom) and a double pendulum (a chaotic 2-degree-of-freedom system). Performance was tested in both "full" and "small" data regimes, with the latter containing significantly fewer observations (e.g., 64 vs 600 points for the single pendulum).
Single Pendulum Results
In the single pendulum experiments, the addition of energy regularization provided a clear advantage. As shown in the results below, the mean energy error ($\Delta E_s$) in the small data regime was reduced by a factor of five when comparing the standard HNN to the regularized version.
Double Pendulum Results
The impact was even more pronounced for the chaotic double pendulum. Standard models often diverged or produced high errors when data was scarce. However, the combination of coordinate constraints (CHHN) and energy regularization (H-Reg) proved exceptionally robust.
Key Insights
- Gradients are Insufficient: Learning only the "slope" of the energy landscape is inadequate in data-starved environments. Without an absolute reference, the model can drift into physically impossible states.
- Energy as an Anchor: Absolute energy levels act as a global constraint, preventing long-term predictive drift.
- CHHN + Regularization: For chaotic systems like the double pendulum, combining structural constraints with energy regularization was the only method that remained stable in the small data regime.
Conclusion and Future Work
While energy-level regularization significantly enhances PINNs, it introduces the hyperparameter $\lambda_H$, which currently requires manual tuning. Furthermore, the method assumes the total energy of training points is known, which may not always be feasible in experimental settings. Future research could explore automated tuning of regularization weights using techniques from the broader PINN literature to make the approach more plug-and-play.
Summary updated