Complexity Constrained Symbolic Regression for Scientific Laws

This direction is split into other directions: multi-objective model selection. Its last version stays readable and citable here.

Version 1 of 1184 papers222 conceptsfrozen 2026-10-04

In the natural sciences, the "holy grail" of data analysis is not just a accurate prediction, but a concise mathematical law. While deep learning models can predict the trajectory of a planet with high precision, they often do so through millions of opaque parameters. In contrast, Johannes Kepler described the same motion with a single, interpretable equation.

Symbolic Regression (SR) is the field dedicated to bridging this gap. It treats the discovery of governing equations as a search problem over the space of mathematical symbols. However, because the number of ways to combine operators ($+$, $\sin$, $\exp$) and variables is infinite, the search space is "bumpy" and prone to overfitting. Modern research focuses on parsimony: the principle that among competing hypotheses that explain the data equally well, the simplest one is usually the right one.

The Pareto Front of Complexity and Accuracy

The fundamental challenge in symbolic regression is the trade-off between how well an equation fits the data (accuracy) and how short the equation is (complexity). If we only optimize for accuracy, the algorithm might produce a massive, "bloated" expression that fits every bit of noise in the dataset.

To solve this, researchers like Silviu-Marian Udrescu and [Max Tegmark] (in AI Feynman 2.0) employ Pareto optimization. Instead of returning a single "best" formula, these systems generate a "Pareto front"—a menu of models where no model is both simpler and more accurate than another. This allows scientists to navigate the trade-off, often finding that the "true" physical law sits at the "elbow" of the curve, where a small increase in complexity yields a massive jump in accuracy.

Sihao Huang and colleagues demonstrated this in OccamNet, showing that by explicitly penalizing structural complexity, AI can rediscover economic and biological laws that remain transparent enough for human experts to validate.

Search Strategies: From Evolution to Trees

How do we actually navigate this infinite sea of equations? Traditional methods used Genetic Programming (GP), evolving populations of equations like biological organisms. However, newer approaches use more structured search techniques:

  1. Monte Carlo Tree Search (MCTS): By treating equation construction as a game—where each "move" is adding a symbol to a tree—researchers like Fangzheng Sun and Hao Sun developed the Symbolic Physics Learner (SPL). MCTS allows the agent to "look ahead" and estimate which branches of the mathematical forest are most likely to lead to a parsimonious solution.
  2. Genetic Forests: Yuntian Chen and Yahong Chen introduced SGA-PDE, which treats equations as forests of binary trees. This allows the discovery of "open-form" partial differential equations (PDEs) without requiring a pre-defined library of terms.

Neural Guidance and the Minimum Description Length

The most recent shift in the field involves using neural networks not as the final model, but as a "compass" to guide the symbolic search.

In DGSR-MCTS, Pierre-Alexandre Kamienny and Guillaume Lample use pre-trained Transformers to suggest smart mutations to equations. Similarly, Hengzhe Zhang introduced RAG-SR, which uses Retrieval-Augmented Generation to ground neural suggestions in real-time mathematical discoveries.

A key conceptual breakthrough here is moving away from minimizing "prediction error" and toward minimizing Minimum Description Length (MDL). As Jingtao Ding and Zihan Yu argue in their work on MDLformer, prediction error is a "noisy" signal—a tiny change in a formula can lead to a huge change in error. Description length, however, provides a smoother, monotonic signal that points toward the simplest, most generalizable formula.

Beyond Physics: Broadening the Scope

While much of this work is rooted in physics, the principles of parsimonious discovery are spreading to other domains:

By enforcing parsimony, these researchers are ensuring that as AI becomes more powerful, the "laws" it discovers remain something humans can actually understand. As Saaketh Desai notes in the work on Parsimonious Neural Networks, the goal is for AI to not just predict mechanics, but to "teach" it back to us.

Papers in this version

Showing 60 of 184 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_complexity_constrained_symbolic_regressi_8cacd186,
  title = {Complexity Constrained Symbolic Regression for Scientific Laws},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complexity-constrained-symbolic-regression-for-scientific-laws/rdr_8cacd186005943a78eb9fd119a7cfd22},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_complexity_constrained_symbolic_regressi_8cacd186_v1,
  title = {Complexity Constrained Symbolic Regression for Scientific Laws},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/complexity-constrained-symbolic-regression-for-scientific-laws/rdr_8cacd186005943a78eb9fd119a7cfd22/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}