Abstract
In fields such as physics and engineering, the goal of modeling is often to discover the underlying mathematical laws governing a system rather than merely achieving high predictive accuracy. This paper introduces GINN-LP, an interpretable neural network architecture designed for Symbolic Regression (SR). GINN-LP specifically targets the discovery of multivariate Laurent polynomials by employing a "growing" architecture and a specialized Power-Term Approximator (PTA) block. By mapping network weights directly to exponents and coefficients, GINN-LP transforms the discrete search for equations into a continuous optimization problem. Experimental results on the Feynman dataset demonstrate that GINN-LP achieves a state-of-the-art solution rate of 87.5%, significantly outperforming existing methods like AIFeynman.
Introduction
Traditional Symbolic Regression methods, such as Genetic Programming, often struggle with the "combinatorial explosion" of possible functional forms as the number of variables increases. Conversely, deep neural networks, while powerful, are typically "black boxes" that offer little physical insight. GINN-LP addresses these challenges by creating a network that is interpretable by design. It focuses on Laurent Polynomials—expressions where variables are raised to any integer power (e.g., $Ax^2y^{-1} + Bz^3$)—which are common in physical laws.
Methodology: The Power-Term Approximator
The core innovation of GINN-LP is the Power-Term Approximator (PTA) block. Standard neural networks use activation functions like ReLU, which obscure the relationship between inputs and outputs. In contrast, the PTA block utilizes a specific sequence of operations to learn exponents through standard backpropagation.
As shown in Figure 1, the PTA block implements the mathematical identity $\exp(w \cdot \log(x)) = x^w$. Here, the weights ($w$) of the linear layer directly represent the exponents of the input variables. Multiple PTA blocks are arranged in parallel, with their outputs summed to represent individual terms in a polynomial.
Growing Strategy and Training
To avoid overfitting and ensure the simplest possible model, GINN-LP employs a growth strategy. The model begins with a single PTA block and adds new blocks (terms) only if the training error remains above a certain threshold. This ensures the discovered equations remain concise. Training involves a two-phase process:
- Sparsity-Promoting Phase: Uses $L_1$ and $L_2$ regularization to encourage the network to use fewer terms and variables.
- Fine-Tuning Phase: Removes regularization to precisely determine the coefficients of the discovered equation.
Additionally, GINN-LP includes an ensemble pipeline. If the model learns non-integer exponents, it indicates the underlying law may not be a Laurent polynomial, triggering a secondary SR method like AIFeynman to handle more complex functional forms.
Experimental Results
The authors evaluated GINN-LP using SRBench, specifically focusing on the Feynman Dataset, which contains 113 datasets derived from classic physics equations.
On a subset of 48 multivariate Laurent polynomial equations, GINN-LP achieved a median solution rate of 87.5%, surpassing AIFeynman's 77.08%. When integrated into an ensemble with AIFeynman for the full 113-equation benchmark, the combined system provided a 7.1% absolute improvement over the previous best-performing method. The model also demonstrated significant robustness to irrelevant input features and performed reliably with as few as 500 data points.
Conclusion
GINN-LP represents a significant step toward making deep learning models physically interpretable. By turning equation discovery into a differentiable optimization task, it circumvents the search space issues of traditional SR. While currently limited to positive inputs and specific functional forms, future work aims to extend the architecture to handle negative values and more complex operations like trigonometry, further bridging the gap between machine learning and scientific discovery.
Summary updated