Abstract
As machine learning models are increasingly deployed to discover physical laws from data, a critical question arises: do different models converge on the same underlying principles, or do they produce a "Tower of Babel" of conflicting theories? We introduce MASS (Multiple AI Scalar Scientists), a framework designed to investigate the convergence of AI-discovered physical descriptions. Our results demonstrate that as physical systems increase in complexity, AI "scientists" naturally discard redundant hypotheses and converge toward a singular, accurate family of descriptions—specifically favoring Lagrangian mechanics over Hamiltonian alternatives.
Introduction
Historically, human scientists have often arrived at different but mathematically equivalent formulations of the same phenomena, such as the independent development of calculus by Newton and Leibniz. In the era of "AI Scientists," where neural networks discover laws from raw observational data, we face the challenge of "multi-expressivity." If two identical networks trained with different initializations find different mathematical expressions for the same data, has the AI found a universal truth or merely a local fit? This work explores whether these models eventually converge on a unified theory and identifies the mathematical "language" they prefer.
The MASS Framework
The authors propose MASS (Multiple AI Scalar Scientists), an architecture that avoids "baking in" specific physical equations, instead allowing them to emerge through a three-stage process:
- Hypothesis Formation: For each system, a neural network learns a scalar function ($S$), representing a physical quantity like energy or action.
- Theory Evaluation: A shared final layer applies derivatives (up to second order) to this scalar function to predict system evolution. This layer acts as the "theory" or mathematical rule set used to interpret the learned scalar.
- Optimization: The model is trained to minimize trajectory prediction error across multiple diverse physical systems simultaneously.
By examining the weights in the final shared layer, researchers can interpret the specific physical theory the AI has "invented."
Experimental Setup
MASS was evaluated on a range of classical mechanics problems:
- Standard 1D Systems: Simple Harmonic Oscillator (SHO), Simple Pendulum, and Keplerian orbits.
- Synthetic Systems: Relativistic oscillators and custom potentials designed to challenge standard frameworks.
- High-Dimensional Chaos: Double Pendulum, Spherical Pendulum, and the N-body problem (including the "figure-8" three-body orbit).
Key Insights: Convergence and the Lagrangian Preference
A primary finding is the "rise and fall" of theories. Initially, models entertain numerous complex or incorrect hypotheses. However, as they encounter more diverse data, the AI is forced to simplify. The number of significant weights in the model decreases, indicating a move toward more fundamental, generalized laws.
Furthermore, while simple problems might allow for Hamiltonian-like descriptions, complex or relativistic systems drive the AI almost universally toward a Lagrangian formulation. This preference likely stems from the robustness of Lagrangian mechanics when dealing with generalized coordinates (e.g., angles) compared to the strict canonical coordinate requirements of Hamiltonian mechanics.
Results
The MASS framework demonstrates high physical fidelity, achieving a Mean Squared Error (MSE) of $3 \times 10^{-4}$ for the simple harmonic oscillator.
Key quantitative findings include:
- Theory Convergence: In 1D relativistic systems, the correlation ($R^2$) to a Lagrangian description reached 0.9999, far outperforming the Hamiltonian fit (0.8205).
- Sparsification: Increasing the number of training systems from 1 to 6 reduced the number of "significant terms" from over 30 to approximately 6, reflecting the discovery of concise physical laws.
- Mathematical Equivalence: Even with different internal weights, the theories produced by different models are often equivalent. Principal component analysis showed that the first component of activations explains over 90% of the variance between models.
Discussion and Future Work
While promising, the MASS approach faces challenges. Training is inherently unstable due to the requirement of computing Hessian matrices and their inverses, which can lead to gradient issues. Additionally, the $O(d^2)$ scaling with dimensionality $d$ limits its application to extremely high-dimensional systems. Future research will focus on integrating symbolic regression to produce human-readable equations and exploring whether AI can learn its own coordinate transformations while maintaining its preference for the Lagrangian framework.
Summary updated