Do Two AI Scientists Agree?

Xinghong Fu, Ziming Liu, Max TegmarkarXiv 2025 · DOI · arXiv

Abstract

As machine learning models are increasingly deployed to discover physical laws from data, a critical question arises: do different models converge on the same underlying principles, or do they produce a "Tower of Babel" of conflicting theories? We introduce MASS (Multiple AI Scalar Scientists), a framework designed to investigate the convergence of AI-discovered physical descriptions. Our results demonstrate that as physical systems increase in complexity, AI "scientists" naturally discard redundant hypotheses and converge toward a singular, accurate family of descriptions—specifically favoring Lagrangian mechanics over Hamiltonian alternatives.

Introduction

Historically, human scientists have often arrived at different but mathematically equivalent formulations of the same phenomena, such as the independent development of calculus by Newton and Leibniz. In the era of "AI Scientists," where neural networks discover laws from raw observational data, we face the challenge of "multi-expressivity." If two identical networks trained with different initializations find different mathematical expressions for the same data, has the AI found a universal truth or merely a local fit? This work explores whether these models eventually converge on a unified theory and identifies the mathematical "language" they prefer.

The MASS Framework

The authors propose MASS (Multiple AI Scalar Scientists), an architecture that avoids "baking in" specific physical equations, instead allowing them to emerge through a three-stage process:

  1. Hypothesis Formation: For each system, a neural network learns a scalar function ($S$), representing a physical quantity like energy or action.
  2. Theory Evaluation: A shared final layer applies derivatives (up to second order) to this scalar function to predict system evolution. This layer acts as the "theory" or mathematical rule set used to interpret the learned scalar.
  3. Optimization: The model is trained to minimize trajectory prediction error across multiple diverse physical systems simultaneously.
The MASS architecture: a three-stage pipeline from raw data to a shared overarching physical theory.

By examining the weights in the final shared layer, researchers can interpret the specific physical theory the AI has "invented."

Experimental Setup

MASS was evaluated on a range of classical mechanics problems:

  • Standard 1D Systems: Simple Harmonic Oscillator (SHO), Simple Pendulum, and Keplerian orbits.
  • Synthetic Systems: Relativistic oscillators and custom potentials designed to challenge standard frameworks.
  • High-Dimensional Chaos: Double Pendulum, Spherical Pendulum, and the N-body problem (including the "figure-8" three-body orbit).

Key Insights: Convergence and the Lagrangian Preference

A primary finding is the "rise and fall" of theories. Initially, models entertain numerous complex or incorrect hypotheses. However, as they encounter more diverse data, the AI is forced to simplify. The number of significant weights in the model decreases, indicating a move toward more fundamental, generalized laws.

Furthermore, while simple problems might allow for Hamiltonian-like descriptions, complex or relativistic systems drive the AI almost universally toward a Lagrangian formulation. This preference likely stems from the robustness of Lagrangian mechanics when dealing with generalized coordinates (e.g., angles) compared to the strict canonical coordinate requirements of Hamiltonian mechanics.

Results

The MASS framework demonstrates high physical fidelity, achieving a Mean Squared Error (MSE) of $3 \times 10^{-4}$ for the simple harmonic oscillator.

Training dynamics and simulation accuracy. (a) The number of significant weights decreases as the model converges. (b) The predicted trajectory closely matches the analytical solution.

Key quantitative findings include:

  • Theory Convergence: In 1D relativistic systems, the correlation ($R^2$) to a Lagrangian description reached 0.9999, far outperforming the Hamiltonian fit (0.8205).
  • Sparsification: Increasing the number of training systems from 1 to 6 reduced the number of "significant terms" from over 30 to approximately 6, reflecting the discovery of concise physical laws.
  • Mathematical Equivalence: Even with different internal weights, the theories produced by different models are often equivalent. Principal component analysis showed that the first component of activations explains over 90% of the variance between models.

Discussion and Future Work

While promising, the MASS approach faces challenges. Training is inherently unstable due to the requirement of computing Hessian matrices and their inverses, which can lead to gradient issues. Additionally, the $O(d^2)$ scaling with dimensionality $d$ limits its application to extremely high-dimensional systems. Future research will focus on integrating symbolic regression to produce human-readable equations and exploring whether AI can learn its own coordinate transformations while maintaining its preference for the Lagrangian framework.

Summary updated

Abstract

When two AI models are trained on the same scientific task, do they learn the same theory or two different theories? Throughout history of science, we have witnessed the rise and fall of theories driven by experimental validation or falsification: many theories may co-exist when experimental data is lacking, but the space of survived theories become more constrained with more experimental data becoming available. We show the same story is true for AI scientists. With increasingly more systems provided in training data, AI scientists tend to converge in the theories they learned, although sometimes they form distinct groups corresponding to different theories. To mechanistically interpret what theories AI scientists learn and quantify their agreement, we propose MASS, Hamiltonian-Lagrangian neural networks as AI Scientists, trained on standard problems in physics, aggregating training results across many seeds simulating the different configurations of AI scientists. Our findings suggests for AI scientists switch from learning a Hamiltonian theory in simple setups to a Lagrangian formulation when more complex systems are introduced. We also observe strong seed dependence of the training dynamics and final learned weights, controlling the rise and fall of relevant theories. We finally demonstrate that not only can our neural networks aid interpretability, it can also be applied to higher dimensional problems.
BibTeX
@misc{fu2025do,
  author = {Xinghong Fu and Ziming Liu and Max Tegmark},
  title = {{Do Two AI Scientists Agree?}},
  year = {2025},
  eprint = {2504.02822},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.02822},
}
Figure