Symbolic Regression (SR)—the task of finding a mathematical formula that fits a dataset—is often described as searching for a needle in a haystack where the haystack is infinite and the needle is made of glass. In a new paper presented at NeurIPS 2026, researchers introduce DASS (Dynamic Auxiliary Search Strategy), a "GPS" for symbolic regression that helps various solvers navigate this massive search space more effectively.
TL;DR
DASS is a plug-and-play guidance module that nearly doubles the accuracy of existing symbolic regression solvers by maintaining a dynamic "subspace" of promising mathematical terms. By iteratively refining these terms based on solver feedback and multi-environment testing, it achieves massive performance gains with only a modest (~24%) increase in runtime.
What Problem Does This Paper Address?
The fundamental challenge in Symbolic Regression is the combinatorial explosion of the search space. Unlike standard linear regression, where the functional form is fixed, SR tries to discover the form itself. The number of ways to combine basic operators ($+, \sin, \exp$, etc.) grows exponentially with the length of the expression.
Most existing solvers—whether based on Genetic Programming (GP), Reinforcement Learning, or Large Language Models (LLMs)—rely on repeated trial-and-error. They often lack a "memory" of which structural components are actually working across different conditions, leading to inefficient searches and a tendency to overfit to local noise.
Methods
DASS doesn't replace existing SR solvers; it sits in front of them as a solver-adaptable auxiliary strategy. Its workflow follows a "virtuous closed-loop":
- Subspace Initialization: DASS uses an LLM to generate a set of "basis terms" (e.g., $x^2$, $\sin(\omega t)$) relevant to the specific scientific domain (Physics, Biology, etc.).
- Multi-Environment Calibration: To ensure these terms are robust, DASS evaluates them across three environments: Extrapolation (testing on data outside the training range), Sparsity (fitting on limited data), and Noise (handling corrupted signals).
- Stability Filtering: Terms that perform well in one environment but fail in others are filtered out. The remaining "high-potential" terms are assigned scores using a Gibbs posterior update.
- Solver Guidance: These terms are fed to the downstream solver. For LLM-based solvers, they are injected into prompts as "structural hints." For GP-based solvers, they bias mutation and crossover operations.
- Subspace Reshaping: High-quality solutions found by the solver are fed back into DASS to update the basis term set, replacing low-utility terms with new, successful ones.
Datasets
The authors evaluated DASS using LLM-SRBench, specifically the LSR-Synth subset. This dataset is designed to avoid "data contamination" (where an LLM might have memorized a famous physics equation during training) by combining known functional terms into novel, synthetic expressions. It covers four domains:
- Physics (44 tasks)
- Materials Science (25 tasks)
- Chemistry (36 tasks)
- Biology (24 tasks)
Key Insights and Ideas
- Terms over Symbols: Instead of guiding solvers at the level of individual symbols (which are too fine-grained) or full equations (which are too rigid), DASS operates on basis terms (subtrees). This is the "Goldilocks" level of abstraction for providing meaningful structural guidance.
- Robustness as a Signal: By testing terms in multiple "hostile" environments (noise, extrapolation), DASS identifies components that represent the true underlying physics rather than just numerical coincidences.
- Universal Compatibility: Because DASS outputs a set of weighted terms, it can be integrated into almost any search paradigm, from discrete MCTS to continuous latent-space searches.
Key Results
The researchers integrated DASS into five representative solvers (LLM-SR, LaSR, PySR, gplearn, and SPL). The results were striking:
- Accuracy Boost: DASS nearly doubled the average accuracy across all solvers, with a 97.84% relative gain in task-solving success.
- Efficiency: DASS-augmented solvers with a "low budget" often outperformed "high budget" versions of the original solvers. For example, LLM-SR with DASS achieved comparable accuracy to the base version while being 8x faster.
- Manageable Overhead: The sophisticated calibration process only added an average of 24.14% to the total search time.
Limitations and Future Work
While DASS is highly adaptable, its integration still requires the downstream solver to have an interface for receiving guidance (e.g., a prompt or a grammar library). For "black-box" solvers, the authors had to intervene at the latent-search or policy level, which requires more engineering effort.
Future work aims to explore even more universal guidance signals and richer subspace representations. The authors also suggest that this closed-loop paradigm—where an auxiliary strategy refines the search space while the solver explores it—could be extended to broader scientific discovery tasks beyond just symbolic regression.
Summary updated