Constants of motion network revisited

Wenqi Fang, Chao Chen, Yongkui Yang, Zheng WangarXiv 2025 · DOI · arXiv

Abstract

Discovering constants of motion—quantities like energy or angular momentum that remain invariant over time—is a cornerstone of physics. While machine learning approaches like the Constant Of Motion nETwork (COMET) have shown promise in identifying these from data, they often struggle with overparameterization and high sensitivity to noise. This paper presents Meta-COMET, an enhanced framework that integrates Singular Value Decomposition (SVD) into the neural network architecture. Meta-COMET reduces model size by up to 14 times and significantly improves robustness to experimental noise, providing a more practical tool for scientific discovery in complex dynamical systems.

Introduction

In the study of dynamical systems, identifying "laws" or invariants is the primary goal. Noether’s theorem guarantees the existence of constants of motion wherever symmetries are present, but extracting these from noisy, real-world data remains a challenge. Previous deep learning attempts, specifically COMET, utilized multi-layer perceptrons (MLPs) to find functions that stay constant during evolution. However, these models are often unnecessarily large and prone to "hallucinating" deviations when faced with even minor data noise. Meta-COMET addresses these issues by enforcing a low-rank structure on the network weights, effectively filtering noise and compressing the model representation.

Methodology: SVD-Inspired Architecture

The core innovation of Meta-COMET lies in its specialized layers: svd-FC and sd-FC. Instead of standard dense weight matrices, these blocks decompose weights into an SVD-like structure: $W \approx S \cdot V \cdot D^T$.

The Meta-COMET architecture utilizing SVD-inspired blocks (SVD-FC and SD-FC) to link state dynamics and constants of motion.

In this architecture, $S$ and $D$ are constrained to be semi-orthogonal, while $V$ is a diagonal matrix. By applying a ReLU activation to the diagonal elements of $V$, the network can automatically truncate less significant values during training. This process performs automated model compression, ensuring the network only retains the most essential physical features. Furthermore, by sharing the $S$ and $D$ matrices between the paths for state dynamics ($\dot{s}_0$) and constants ($c$), the model mathematically enforces the intrinsic link between a system's motion and its conservation laws.

Experimental Validation

The researchers evaluated Meta-COMET on a wide range of systems, including:

  • Classical Mechanics: Mass-spring systems, 2D pendulums, and the two-body problem.
  • Biological Systems: The Lotka-Volterra predator-prey model.
  • Partial Differential Equations: The Korteweg-De Vries (KdV) equation.
  • Visual Data: Learning dynamics directly from noisy pixel representations of physical systems.

Key Results and Discussion

Meta-COMET demonstrated significant improvements over the baseline COMET in two critical areas: efficiency and stability.

Massive Model Compression

By leveraging the low-rank SVD structure, Meta-COMET achieved substantial parameter reduction. For instance, in the mass-spring and Lotka-Volterra tasks, the model was 14 times smaller than the original COMET. Even for complex systems like the KdV equation, Meta-COMET required only ~56k parameters compared to COMET's ~628k, without sacrificing accuracy.

Superior Noise Robustness

Perhaps the most significant advantage is the model's resilience to noise. The low-rank constraint acts as a natural regularizer, forcing the network to ignore high-frequency noise that does not align with the underlying physical manifold.

Comparison of predicted physical invariants over time under noisy conditions for various systems. Meta-COMET remains stable while COMET deviates significantly from the ground truth.

As shown in the experimental results, when subjected to noise levels of $\sigma = 0.2$, the original COMET's predictions for constants like energy began to oscillate wildly. In contrast, Meta-COMET's predictions remained stable and closely tracked the ground truth across multiple time steps and system types.

Conclusion

Meta-COMET represents a significant step forward in making physics-informed AI more efficient and robust. By embedding fundamental linear algebra principles like SVD into the neural architecture, the authors have created a model that is not only smaller but also more scientifically grounded. Future work will focus on providing rigorous mathematical proofs for this observed noise immunity and comparing the framework against other emerging methods in the field of scientific machine learning.

Summary updated

BibTeX
@misc{fang2025constants,
  author = {Wenqi Fang and Chao Chen and Yongkui Yang and Zheng Wang},
  title = {{Constants of motion network revisited}},
  year = {2025},
  eprint = {2504.09434},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.09434},
}
Figure