Abstract
Discovering constants of motion—quantities like energy or angular momentum that remain invariant over time—is a cornerstone of physics. While machine learning approaches like the Constant Of Motion nETwork (COMET) have shown promise in identifying these from data, they often struggle with overparameterization and high sensitivity to noise. This paper presents Meta-COMET, an enhanced framework that integrates Singular Value Decomposition (SVD) into the neural network architecture. Meta-COMET reduces model size by up to 14 times and significantly improves robustness to experimental noise, providing a more practical tool for scientific discovery in complex dynamical systems.
Introduction
In the study of dynamical systems, identifying "laws" or invariants is the primary goal. Noether’s theorem guarantees the existence of constants of motion wherever symmetries are present, but extracting these from noisy, real-world data remains a challenge. Previous deep learning attempts, specifically COMET, utilized multi-layer perceptrons (MLPs) to find functions that stay constant during evolution. However, these models are often unnecessarily large and prone to "hallucinating" deviations when faced with even minor data noise. Meta-COMET addresses these issues by enforcing a low-rank structure on the network weights, effectively filtering noise and compressing the model representation.
Methodology: SVD-Inspired Architecture
The core innovation of Meta-COMET lies in its specialized layers: svd-FC and sd-FC. Instead of standard dense weight matrices, these blocks decompose weights into an SVD-like structure: $W \approx S \cdot V \cdot D^T$.
In this architecture, $S$ and $D$ are constrained to be semi-orthogonal, while $V$ is a diagonal matrix. By applying a ReLU activation to the diagonal elements of $V$, the network can automatically truncate less significant values during training. This process performs automated model compression, ensuring the network only retains the most essential physical features. Furthermore, by sharing the $S$ and $D$ matrices between the paths for state dynamics ($\dot{s}_0$) and constants ($c$), the model mathematically enforces the intrinsic link between a system's motion and its conservation laws.
Experimental Validation
The researchers evaluated Meta-COMET on a wide range of systems, including:
- Classical Mechanics: Mass-spring systems, 2D pendulums, and the two-body problem.
- Biological Systems: The Lotka-Volterra predator-prey model.
- Partial Differential Equations: The Korteweg-De Vries (KdV) equation.
- Visual Data: Learning dynamics directly from noisy pixel representations of physical systems.
Key Results and Discussion
Meta-COMET demonstrated significant improvements over the baseline COMET in two critical areas: efficiency and stability.
Massive Model Compression
By leveraging the low-rank SVD structure, Meta-COMET achieved substantial parameter reduction. For instance, in the mass-spring and Lotka-Volterra tasks, the model was 14 times smaller than the original COMET. Even for complex systems like the KdV equation, Meta-COMET required only ~56k parameters compared to COMET's ~628k, without sacrificing accuracy.
Superior Noise Robustness
Perhaps the most significant advantage is the model's resilience to noise. The low-rank constraint acts as a natural regularizer, forcing the network to ignore high-frequency noise that does not align with the underlying physical manifold.
As shown in the experimental results, when subjected to noise levels of $\sigma = 0.2$, the original COMET's predictions for constants like energy began to oscillate wildly. In contrast, Meta-COMET's predictions remained stable and closely tracked the ground truth across multiple time steps and system types.
Conclusion
Meta-COMET represents a significant step forward in making physics-informed AI more efficient and robust. By embedding fundamental linear algebra principles like SVD into the neural architecture, the authors have created a model that is not only smaller but also more scientifically grounded. Future work will focus on providing rigorous mathematical proofs for this observed noise immunity and comparing the framework against other emerging methods in the field of scientific machine learning.
Summary updated