Identifying Physical Law of Hamiltonian Systems via Meta-Learning

Seungjun Lee, Haesang Yang, Woojae Seong

Introduction

In physics, a Hamiltonian is a mathematical function that describes the total energy of a system. If you know the Hamiltonian, you can accurately predict how the system will evolve over time. While the formulas for simple systems like pendulums have been known for centuries, identifying the Hamiltonian for complex, unknown systems directly from observational data remains incredibly difficult.

Current deep learning approaches, such as Hamiltonian Neural Networks (HNNs), excel at learning the dynamics of a single system. However, they struggle significantly with generalization. For instance, if you train an HNN on a 1-meter pendulum, it cannot accurately predict the behavior of a 2-meter pendulum without being retrained from scratch.

This paper addresses the challenge of identifying the shared physical law across a family of related systems. By combining HNNs with meta-learning, the researchers have created a model that learns the underlying physical law itself, allowing it to adapt to new parameters (like mass or length) using only a tiny amount of observational data.

Conceptual diagram of meta-learning applied to Hamiltonian systems.

Methodology

The authors propose two main algorithms that merge physics-informed neural network architectures with meta-learning techniques:

  1. HAMAML (Hamiltonian Model-Agnostic Meta-Learning): Based on the popular MAML algorithm, this approach treats each physical system (e.g., a specific pendulum with unique mass and length) as a distinct "task." During meta-training, the model learns a set of initial weights that can be quickly updated via gradient descent to fit any new system within that family.
  2. HANIL (Hamiltonian Almost No Inner Loop): This is a more efficient variant of HAMAML. It relies on the "feature reuse hypothesis," which suggests that the "body" of a neural network learns the general features (the overarching physical law), while only the "head" (the final layer) needs to change to adapt to specific parameters (constants like mass $m$ or length $l$).

By utilizing the HNN architecture, both models are forced to respect the symplectic structure of Hamiltonian mechanics. This crucial inductive bias ensures the networks learn to conserve energy, rather than merely curve-fitting the training data.

Datasets and Experiments

The researchers evaluated their methods on three classic synthetic physics environments:

  • Spring-Mass System: A simple harmonic oscillator where the mass, spring constant, and equilibrium position vary.
  • Pendulum: A standard nonlinear pendulum where mass, length, and equilibrium angle vary.
  • Kepler Problem: A two-body planetary motion problem (similar to a planet orbiting a star) where the masses of the bodies vary.

For each environment, they generated 10,000 different "tasks" (systems with different physical parameters) for meta-training, and subsequently tested the models on 10 entirely new, unseen systems.

Key Insights and Results

The core insight of the paper is that "identifying a physical law" is conceptually identical to "meta-learning." A physical law serves as a shared representation across tasks, while specific physical constants act as task-specific parameters.

The experimental results were decisive across all three physical systems:

  • The Power of HANIL: Surprisingly, HANIL (which only updates the last layer during adaptation) significantly outperformed HAMAML (which updates all weights). This suggests that the neural network successfully "hard-coded" the functional form of the physical law into its internal layers, leaving only the specific coefficients to be adjusted in the final layer.
  • Extreme Data Efficiency: The meta-learned model can identify a new system's behavior from as few as 25 data points (shots). A standard HNN would require thousands of points and significantly more training time to achieve comparable results.
  • Accuracy and Energy Conservation: HANIL achieved Mean Squared Errors (MSE) that were often 10 to 100 times lower than baselines like "HNN from scratch" or "Pretrained HNN." For the Spring-Mass system, HANIL achieved an MSE of 0.02, compared to 8.2 for a standard pretrained HNN. Qualitatively, the meta-learned models produced phase portraits that perfectly matched the ground truth, maintaining the closed loops that represent perfectly conserved energy.
  • Rapid Adaptation: While a standard HNN requires thousands of gradient steps to learn a system, the meta-learner reached peak performance in just 10 to 50 steps.
Phase portraits comparing adaptation across different methods and data sizes.

Limitations and Future Work

While promising, the study relies entirely on simulated, noise-free data. Real-world sensor data is inherently messy, which might challenge the model's ability to identify the underlying symplectic structure. Furthermore, while the Kepler problem introduces complexity, these remain relatively low-dimensional systems. Scaling this approach to high-dimensional problems like fluid dynamics or many-body quantum systems remains an open challenge.

Finally, despite acting as though it "knows" the physical law, the model remains a black box and does not output a human-readable mathematical formula. Future work could involve combining this meta-learning approach with symbolic regression to extract actual equations directly from the adapted network weights.

Summary updated

Abstract

Hamiltonian mechanics is an effective tool to represent many physical processes with concise yet well-generalized mathematical expressions. A well-modeled Hamiltonian makes it easy for researchers to analyze and forecast many related phenomena that are governed by the same physical law. However, in general, identifying a functional or shared expression of the Hamiltonian is very difficult. It requires carefully designed experiments and the researcher's insight that comes from years of experience. We propose that meta-learning algorithms can be potentially powerful data-driven tools for identifying the physical law governing Hamiltonian systems without any mathematical assumptions on the representation, but with observations from a set of systems governed by the same physical law. We show that a well meta-trained learner can identify the shared representation of the Hamiltonian by evaluating our method on several types of physical systems with various experimental settings.
BibTeX
@inproceedings{lee2021identifying,
  author = {Seungjun Lee and Haesang Yang and Woojae Seong},
  title = {{Identifying Physical Law of Hamiltonian Systems via Meta-Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2021},
  url = {https://openreview.net/forum?id=K7Yj2qXmVm},
}
Figure