Automated Symbolic Law Discovery: A Computer Vision Approach

Hengrui Xing, Ansaf Salleb-Aouissi, Nakul VermaAAAI 2021 · OpenReview

Abstract

One of the most exciting applications of modern artificial intelligence is to automatically discover scientific laws from experimental data. This is not a trivial problem as it involves searching for a complex mathematical relationship over a large set of explanatory variables and operators that can be combined in an infinite number of ways. Inspired by the incredible success of deep learning in computer vision, we tackle this problem by adapting various successful network architectures into the symbolic law discovery pipeline. The novelty of our approach is in (1) encoding the input data as an image with super-resolution, (2) developing an appropriate deep network pipeline, and (3) predicting the importance of each mathematical operator from the relationship image. This allows us to prior the exponentially large search with the predicted importance of the symbolic operators, which can significantly accelerate the discovery process. We apply our model to a variety of relationships---both simulated and from physics and mathematics domains---involving different dimensions and constituents. We show that our model is able to identify the underlying operators from data, achieving a high accuracy and AUC (91% and 0.96 on average resp.) for systems with as many as ten independent variables. Our method significantly outperforms the current state of the art in terms of data fitting ($R^2$), discovery rate (recovering the true relationship), and succinctness (output formula complexity). The discovered equations can be seen as first drafts of scientific laws that can be helpful to the scientists for hypothesis building and understanding the complex underlying structure of the studied phenomena.

Introduction

In the quest to automate scientific discovery, researchers use Symbolic Regression (SR) to find the simplest mathematical expression that fits a dataset. Unlike standard regression, which finds parameters for a fixed curve, SR searches through an infinite space of possible functional forms. The primary bottleneck in this process is the operator set. To find a law like $F = ma$, an algorithm must know to look for multiplication. If the search space includes too many irrelevant operators (e.g., cosh, bessel_j), the complexity explodes, leading to overfitting. Conversely, too few operators prevent the discovery of the true relationship.

Presented at AAAI-21, the DeSTrOI (Deep Symbolic Tree Operator Identifier) framework addresses this by treating numerical data as an image. By "seeing" the visual fingerprints of mathematical operators, the model can automatically prune the search space, significantly accelerating the discovery of scientific laws while producing more accurate and succinct formulas.

Methodology: The DeSTrOI Pipeline

The authors propose a multi-stage pipeline that leverages the strengths of convolutional neural networks (CNNs) in pattern recognition to identify mathematical structures.

The DeSTrOI architecture: an encoder-decoder pipeline that transforms numerical data into images to identify mathematical operators.

1. Image Encoding and Super-Resolution

The process begins by transforming raw $(x, y)$ data points into a 2D "naive" image by plotting the relationship on a grid. However, experimental data is often sparse or noisy, resulting in blurry visual representations. To combat this, DeSTrOI employs a Super-Resolution Residual Network (SR-ResNet). This network acts as a sophisticated interpolator and denoiser, filling in gaps to reconstruct a high-quality visual manifold of the underlying function.

Comparison of encoding methods: (a) ground truth, (b) naive encoding, (c) super-resolution reconstruction, and (d) MLP baseline.

As shown in the figure above, the super-resolution approach (c) significantly outperforms naive encoding (b) and standard MLP baselines (d) in recovering the sharp features of the ground truth (a), which is critical for identifying complex operators like high-frequency sines.

2. CNN Decoding and Operator Prediction

A VGG-style CNN analyzes the sharpened image to predict the probability that specific operators (e.g., add, multiply, log, sine) are present. For datasets with more than two independent variables, the model generates 2D projections of all variable pairs. An Attention-Based Multiple Instance Learning (MIL) mechanism then determines which projections are most informative, allowing the model to scale to systems with up to ten variables.

Experimental Results

The model was evaluated on thousands of synthetic symbolic trees and classic physics equations, such as the Simple Harmonic Oscillator and the Tsiolkovsky Rocket Equation.

  • High Accuracy: DeSTrOI achieved an average of 91% accuracy and a 0.96 AUC in operator identification across varying dimensions.
  • Enhanced Discovery: Integrating DeSTrOI as a "prior" for Genetic Programming tools led to a 25% increase in the discovery rate of true underlying relationships and a 10% improvement in fit quality ($R^2$).
  • Succinctness and Efficiency: The resulting formulas were 10% less complex, making them more interpretable. Furthermore, DeSTrOI provided speedups of up to 55% for state-of-the-art methods like AI Feynman.

Conclusion

By reframing symbolic regression as a computer vision problem, DeSTrOI offers a powerful way to constrain the search for scientific laws. While currently limited by a fixed operator set and 2D projection methods, the approach demonstrates that deep learning can effectively "see" the underlying structure of physical phenomena, providing scientists with a robust tool for automated hypothesis building and discovery.

Summary updated

Abstract

One of the most exciting applications of modern artificial intelligence is to automatically discover scientific laws from experimental data. This is not a trivial problem as it involves searching for a complex mathematical relationship over a large set of explanatory variables and operators that can be combined in an infinite number of ways. Inspired by the incredible success of deep learning in computer vision, we tackle this problem by adapting various successful network architectures into the symbolic law discovery pipeline. The novelty of our approach is in (1) encoding the input data as an image with super-resolution, (2) developing an appropriate deep network pipeline, and (3) predicting the importance of each mathematical operator from the relationship image. This allows us to prior the exponentially large search with the predicted importance of the symbolic operators, which can significantly accelerate the discovery process. We apply our model to a variety of plausible relationships---both simulated and from physics and mathematics domains---involving different dimensions and constituents. We show that our model is able to identify the underlying operators from data, achieving a high accuracy and AUC (91% and 0.96 on average resp.) for systems with as many as ten independent variables. Our method significantly outperforms the current state of the art in terms of data fitting (R^2), discovery rate (recovering the true relationship), and succinctness (output formula complexity). The discovered equations can be seen as first drafts of scientific laws that can be helpful to the scientists for (1) hypothesis building, and (2) understanding the complex underlying structure of the studied phenomena. Our approach holds a real promise to help speed up the rate of scientific discovery.
BibTeX
@inproceedings{xing2021automated,
  author = {Hengrui Xing and Ansaf Salleb-Aouissi and Nakul Verma},
  title = {{Automated Symbolic Law Discovery: A Computer Vision Approach}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  url = {https://openreview.net/forum?id=qhb2LkY6JKd},
}
Figure