Symbolic regression is the search for a short, human-readable formula that explains data, as opposed to a flexible curve that merely fits it. Its benchmarks score a formula the way old autonomous-driving simulators scored a car. They feed it recorded inputs and check its outputs, and they never let those outputs change what happens next. Driving simulation learned that this kind of "log replay" is misleading. A background car that ignores everyone else causes collisions that would never happen, and its path drifts into nonsense over a long rollout. We argue that equation discovery has the same blind spot. A car-following formula found from noisy vehicle tracks can match each recorded acceleration well and still send velocities negative, blow up in finite time, or erase stop-and-go waves once every car in a platoon obeys it and reacts to the others.
We propose a benchmark and a matched study that score discovered car-following laws at three levels of feedback. The first is one-step fit on logged inputs, which is today's standard. The second is a single-car rollout behind a replayed, non-reactive leader. The third is a fully reactive multi-car rollout on a ring road and in a platoon. Noise models move from the independent Gaussian jitter used in symbolic regression benchmarks to the correlated jitter and dropouts typical of perception tracking. The central claim is testable. Under realistic tracking noise, the ranking of discovery methods by open-loop fit or exact recovery will disagree substantially with their ranking by closed-loop failure. The cause is that noise pushes symbolic search to add small, spurious terms that barely change one-step error but compound under feedback.
What prior work establishes and what it leaves open
Three groups of papers motivate this work. The first concerns evaluation. Contemporary Symbolic Regression Methods and their Relative Performance built SRBench over 252 problems and found two splits: the best black-box fitter (Operon) is not the best recoverer of true formulas, and AI Feynman's 53% low-noise solution rate collapses above 1% noise. SRSD showed that realistic variable ranges cut DSR's hard-set solution rate from 30.0% to 8.00%. SRBench++ found that noise makes models "hallucinate complexity." All three score static mappings with additive Gaussian noise. None asks what a discovered formula does when its outputs become its own future inputs.
The second group concerns noise. Causal chambers showed that methods which rediscover laws under synthetic Gaussian noise fail on real sensor streams. NewtonBench showed that 0.0001 Gaussian noise costs language-model discovery agents 12–16% accuracy. Two methods attack noise directly. EDCDE moves differentiation onto smooth test functions so noisy positions are never differenced, though its reported runs use zero noise. Symmetry-Informed Governing Equation Discovery constrains the search with symmetries. We reuse both as methods under test rather than proposing a competing discovery algorithm.
The third group concerns simulation. The Waymo Open Sim Agents Challenge, TrafficSim and Improving Agent Behaviors with RL Fine-tuning established that agents must be judged in closed loop, but they use opaque neural policies. Limitations and Improvements of the IDM constrains our design. It shows that the standard Intelligent Driver Model (IDM, the hand-written car-following equation in most traffic simulators) allows negative velocities and finite-time blow-ups near standstill. Even a perfectly recovered textbook formula can therefore fail closed-loop. For that reason the benchmark's ground truths are the well-posed variants that paper proposes, not raw IDM.
The closest connecting works each stop short of this question. Closed-form congestion control via deep symbolic regression distills a network-control policy into formulas but does not study discovery from noisy logs or multi-agent feedback. Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation and Differentiable Hybrid Traffic Simulation assume IDM is given rather than discovering it.
Core Claim
Exact recovery and open-loop fit are poor proxies for whether a discovered dynamical law is usable. The gap widens with realistic, correlated noise, and reactive multi-agent rollout exposes it while single-agent replay does not. If this holds, symbolic regression benchmarks for dynamical systems should add closed-loop scoring. A cheap fix should also recover most of the lost reliability: picking among each method's candidate formulas by rollout stability rather than by fit.
Research Program
We build a ground-truth traffic generator. Four to six well-posed car-following laws, drawn from the Albeaik variants, are each paired with "counterfactual" perturbed versions so that language-model methods cannot answer from memory, following NewtonBench's device. The laws drive platoons and ring roads whose true trajectories we record.
The trajectories are then observed through a noise ladder. Its rungs are independent Gaussian position noise at SRBench's levels (0.001, 0.01, 0.1), autocorrelated position error, bounding-box jitter coupled to heading, and track dropouts with interpolation. Each method receives position logs only, so each must handle differentiation itself.
Every discovered formula is scored three ways. Level one is open-loop acceleration $R^2$ together with exact symbolic recovery checked algebraically, as in SRBench. Level two is a 60-second rollout of one follower behind the replayed true leader, measuring gap error and safety violations. Level three replaces every car on the ring road and in the platoon with the discovered law. It measures the fraction of runs with negative velocity, collision, or divergence. It also measures the error in stop-and-go wave amplitude and frequency against the true law's rollouts under identical initial perturbations.
The primary statistic is the Kendall rank correlation between method rankings at level one and at level three, computed per noise rung.
A real-data track extracts car-following pairs from the Waymo Open Motion Dataset. Discovered laws control the longitudinal motion of those followers, while lateral motion stays on the lane centerline. These scenarios are scored with the WOSAC realism metrics. Because this covers only a subset of agents and behaviors, its scores are not comparable to full-scene leaderboard numbers, and we will say so.
Baselines
The naive reference points are log replay, which is non-reactive by construction, and a constant-velocity follower. Any discovered law that does worse than these in closed loop is unusable. Each is run on the identical scenarios.
The classical baseline is IDM and its well-posed variants, with parameters fitted by CMA-ES under the same wall-clock budget given to symbolic search. When the ground truth is an IDM variant, this baseline is an oracle on structure. Its role is to show how much structural discovery costs relative to knowing the form. On the real data, beating it on level-three wave error and WOSAC kinematic and interaction metrics is the bar for discovery being worth doing.
The symbolic methods are PySR, DSR/uDSR, Operon, AI Feynman, gplearn, plain SINDy, symmetry-constrained SINDy using translation invariance in position, and a weak-form variant in the spirit of EDCDE. All run with the same evaluation budget, operator set, and hyperparameter search on a held-out validation split. The strongest reported figures in the file are AI Feynman's 53% low-noise solution rate on SRBench, and symmetry-informed SINDy's 90% success versus 15% for plain SINDy on a damped oscillator at 20% noise (Symmetry-Informed Governing Equation Discovery). No reported result was found on Lacuna for symbolic recovery of car-following laws from tracked vehicle data.
The neural reference is a small learned car-following policy trained by behavior cloning and then fine-tuned closed-loop, in the spirit of the MotionLM REINFORCE work. That work raised its WOSAC collision score from 0.673 to 0.877. Full-scene WOSAC reports 0.722 for the logged oracle and about 0.645 for the best learned models. Those numbers do not transfer to our subset; we cite them only as context.
The key ablation isolates rollout-aware selection. It compares each method's formula picked by validation fit against the formula picked from the same candidate set by level-two or level-three stability. The ablation counts as a success if selection by stability cuts closed-loop failure without raising level-one error by more than a small margin.
Why not just…?
Why not just smooth the trajectories before regression? Savitzky–Golay and Gaussian smoothing are added as preprocessing arms for every method. EDCDE argues that filters either keep derivative spikes or flatten hard-braking events. Causal chambers shows that filtered pipelines still break under physically irrelevant sensor shifts. Smoothing is nonetheless a real competitor. If smoothed PySR matches the weak-form and symmetry methods at level three, we report that simple preprocessing suffices.
Why not just calibrate IDM? It is a baseline above. Its known blow-ups (Limitations and Improvements of the IDM) are structural, so calibration cannot fix them. The well-posed variants, however, may be good enough on real data. In that case the honest conclusion is that discovery adds interpretability but no closed-loop benefit over a fixed, corrected form.
Why not just score multi-step error against a replayed leader? This is the most serious threat, because level two is far cheaper than level three. Car-following instability is collective: string instability amplifies a disturbance along the platoon. A single follower behind a fixed leader cannot amplify anything, which is why we expect level two to miss failures that level three catches. The study tests this directly through the rank correlation between levels two and three. If that correlation exceeds about 0.9 across noise rungs, we will recommend the cheaper protocol and drop the reactive requirement.
Why not just use neural sim agents and skip symbolic discovery? Neural agents need closed-loop fine-tuning to fix the drift caused by training on logged states (Improving Agent Behaviors with RL Fine-tuning). They also lack the analytical gradients that make differentiable traffic simulators fast (Differentiable Hybrid Traffic Simulation). Our neural reference measures how much realism a formula gives up in exchange for those properties.
Proposed Experiments and Decisive Checks
The first decisive check is the central claim. On synthetic data under correlated tracking noise, the Kendall correlation between level-one and level-three rankings should fall well below its value under no noise. In the noiseless setting we expect high agreement, because exactly recovered well-posed laws should also be stable. A correlation that stays high at every noise rung refutes the claim.
The second check is mechanistic. Formulas that pass level one but fail level three should contain extra small terms that the true law lacks, appearing more often as noise increases. We test this by pruning those terms and re-running the rollouts.
The third check is the level-two versus level-three correlation described above.
The fourth check is practical. Rollout-aware selection should lower closed-loop failure rates on the real Waymo car-following subset, and the selected laws should be compared against calibrated well-posed IDM on WOSAC kinematic and interaction metrics.
All results are reported per noise rung, with seeds and failure examples released alongside the generator.
References
- Contemporary Symbolic Regression Methods and their Relative Performance
- SRSD: Rethinking Datasets of Symbolic Regression for Scientific Discovery
- SRBench++: Principled Benchmarking of Symbolic Regression With Domain-Expert Interpretation
- Causal chambers as a real-world physical testbed for AI methodology
- NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents
- EDCDE - Extended Discovery of Closed-Form Differential Equations
- Symmetry-Informed Governing Equation Discovery
- The Waymo Open Sim Agents Challenge
- TrafficSim: Learning To Simulate Realistic Multi-Agent Behaviors
- Improving Agent Behaviors with RL Fine-tuning for Autonomous Driving
- Limitations and Improvements of the Intelligent Driver Model (IDM)
- Closed-form congestion control via deep symbolic regression
- Traffic-Aware Autonomous Driving with Differentiable Traffic Simulation
- Differentiable Hybrid Traffic Simulation