Noise-Resilient Symbolic Regression with Dynamic Gating Reinforcement Learning

Chenglu Sun, Shuo Shen, Wenzhi Tao, Deyi Xue, Zixia ZhouAAAI 2025 · DOI

Abstract

Symbolic Regression (SR) aims to discover mathematical expressions that best fit a given dataset. While powerful, SR often struggles with high-dimensional data containing irrelevant "noise" variables. This paper introduces Noise-Resilient Symbolic Regression (NRSR), a novel framework that integrates a dynamic gating mechanism with reinforcement learning to filter out distractors and explore diverse mathematical structures. NRSR achieves state-of-the-art performance on noisy benchmarks, maintaining high recovery rates where traditional methods fail.

Introduction

Symbolic Regression (SR) is a fundamental task in interpretable machine learning, seeking to uncover the underlying mathematical laws governing a dataset. Unlike black-box neural networks, SR provides transparent, human-readable formulas. However, the search space for these formulas grows exponentially with the number of variables, a problem exacerbated by real-world "noisy" datasets containing irrelevant features. Existing methods often fail to distinguish signal from noise, leading to the discovery of overly complex or incorrect expressions.

The NRSR Framework

NRSR addresses these challenges through two key innovations integrated into a Reinforcement Learning (RL) loop:

  1. Noise-Resilient Gating Module (NGM): This module uses a lightweight gating layer with $L_0$ regularization to perform feature selection before the search process. By identifying and masking irrelevant variables, NGM significantly reduces the effective search space for the reinforcement learning agent.
  2. Mixed Path Entropy (MPE) Regularization: To prevent the RL agent from converging prematurely to local optima, MPE introduces a path-level entropy bonus. Unlike standard entropy which encourages local token diversity, MPE rewards diversity in the entire sequence of tokens, promoting the exploration of fundamentally different mathematical structures.
Architecture of the NRSR framework, showing the Noise-Resilient Gating Module and the RL-based expression generation process.

The core of NRSR is a Recurrent Neural Network (RNN) policy optimized via Proximal Policy Optimization (PPO). The RNN generates pre-order traversals of expression trees, which are then evaluated against the dataset to provide a reward signal based on the Normalized Root Mean Square Error (NRMSE).

Experimental Evaluation

The framework was evaluated using the Nguyen benchmark suite, a standard set of 12 mathematical expressions. To test noise resilience, the researchers created variants of these datasets by adding 5, 10, and 20 irrelevant variables. Performance was measured primarily by the Recovery Rate (RR)—the percentage of trials where the exact symbolic expression was found.

Results and Discussion

NRSR demonstrates significant improvements over existing state-of-the-art methods, particularly in high-noise scenarios. As shown in the results table below, NRSR maintains a high Recovery Rate even as the number of distractor variables increases, while baseline methods see sharp declines in performance.

Comparison of NRSR against baseline methods on datasets with 5 and 10 noisy variables.

Key Findings:

  • Robustness to Distractors: On datasets with 10 noisy variables, NRSR maintained an 89.1% recovery rate, whereas the next best deep learning baseline (DGSR) dropped to 68.5%, and standard DSR plummeted to 23.2%.
  • Search Efficiency: The gating mechanism allows NRSR to find correct expressions with significantly fewer total evaluations (423K vs. over 1.6M for some baselines), proving that the reduced search space leads to faster convergence.
  • Exploration Benefits: Even on clean data, NRSR achieved an 89.7% RR, outperforming all five baselines. This suggests that the MPE exploration bonus is beneficial even in ideal data conditions by preventing the agent from getting stuck in local optima.

Conclusion

NRSR provides a robust solution for symbolic regression in the presence of irrelevant variables. By combining explicit feature gating with path-level entropy regularization, it effectively navigates complex search spaces. Future research may explore its application to higher-dimensional scientific data and its ability to handle Gaussian noise in target values.

Summary updated

BibTeX
@inproceedings{sun2025noise,
  author = {Chenglu Sun and Shuo Shen and Wenzhi Tao and Deyi Xue and Zixia Zhou},
  title = {{Noise-Resilient Symbolic Regression with Dynamic Gating Reinforcement Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i19.34280},
}
Figure