Introduction
Symbolic regression has proven to be a powerful tool for discovering physical laws, such as those found in the AI Feynman project. However, these algorithms typically require clean, structured data tables—a luxury not often found in raw observational science. In the real world, we observe phenomena through high-dimensional pixel data, such as a video of a rocket launch, rather than direct coordinates.
This paper introduces "pregression" (pre-regression), an unsupervised machine learning framework designed to bridge this gap. Pregression automatically extracts the relevant physical degrees of freedom from raw, unlabeled, and often distorted video, preparing the data for standard symbolic regression to identify the underlying equations of motion.
Methodology
The core of the pregression framework is an architecture consisting of three specialized neural networks: an Encoder ($E$), a Decoder ($D$), and an Evolution Operator ($U$). The encoder maps high-dimensional video frames into a low-dimensional latent space, while the decoder ensures the representation remains faithful by reconstructing the original images. The evolution operator $U$ is responsible for predicting future latent states based on previous ones, effectively modeling the dynamics of the system.
Overcoming the "Alexander Principle"
A significant challenge in representation learning is what the authors call the "Alexander Principle": neural networks naturally tend to find highly entangled and overly complex representations if only reconstruction accuracy is optimized. To force the network toward physically interpretable latent spaces, the authors introduce a custom loss function that penalizes complexity. This includes terms that minimize the "curvature" of the evolution function (the Jacobian of $U$) and the acceleration of the latent particles.
Topological Untangling
Mapping high-dimensional image data to a low-dimensional physical space often results in topological defects. To resolve this, the authors leverage a concept from knot theory: knots cannot exist in sufficiently high dimensions. By initially training the autoencoder with a 5D latent space—providing a "topological release valve"—and then projecting down to 2D using Principal Component Analysis (PCA), the model can untangle complex representations that would otherwise get stuck in local minima.
Experimental Setup and Robustness
The model was evaluated on simulated 64x64 pixel videos of a rocket moving under various physical regimes, including force-free motion, gravity, magnetic fields, and harmonic/quartic oscillators. To test the limits of the system, the researchers applied two types of data degradation:
- Spatial Warping: Severe non-linear warping of the video frames.
- Visual Noise: The addition of up to 10 random "distractor" rockets moving independently in the background.
Results and Key Insights
The pregression framework successfully recovered the exact differential equations for all five physical systems, even when faced with significant distortion.
- Robustness: The encoder/decoder networks learned to "absorb" the complex warping functions, keeping the latent space linear and simple. In noisy environments, the model learned to ignore distractors, focusing only on the dynamically consistent main object.
- Inertial Frame Discovery: The algorithm effectively auto-discovers standard isotropic inertial frames. By applying affine transformations to minimize symbolic complexity, it finds the most mathematically elegant coordinate system for the discovered laws.
Conclusion
While currently demonstrated on 2D simulations, pregression offers a promising path toward fully automated scientific discovery from raw observational data. The combination of physics-inspired complexity penalties and topological tricks allows AI to not just see the world, but to understand the simple mathematical rules governing its motion. Future work aims to extend this to 3D dynamics and explore whether biological vision systems utilize similar complexity-minimizing representations.
Summary updated