Scientists increasingly try to recover the governing equations of a system, such as the differential equation behind a predator–prey cycle or a reaction rate, directly from measured time series. The standard tool, sparse regression over a library of candidate terms, works well on clean data. On noisy data it can confidently select terms that only fit measurement noise. A domain expert can often rule such terms out in seconds ("there is no cubic self-interaction in this chemistry"), but expert time is scarce and expert judgments are themselves sometimes wrong. This proposal builds a discovery loop that spends a small, fixed budget of expert questions only on the parts of the equation the noisy data genuinely cannot settle. It treats each answer as fallible evidence, not ground truth.
The central technical point is that the obvious way to find "where the data cannot decide" is broken. A single noisy regression is overconfident about its spurious terms, so ranking terms by how uncertain that regression is will miss the very errors an expert should catch. We propose to measure undecidability as disagreement across different noise-handling formulations of the same regression: pointwise derivatives, weak (integrated) forms, and forward-simulation checks. Spurious noise-fitting terms tend to be confident in one formulation and absent in another. A Bayesian model then combines these formulations with probabilistic expert answers and picks the next question by its expected information per unit cost.
Core Claim
Under heavy measurement noise, a term that one formulation includes with high confidence but another formulation excludes is the signature of a "confidently wrong" selection. Pooling the formulations into one mixture posterior over which terms are present turns that hidden error into visible uncertainty. Given this mixture, choosing expert questions by expected reduction in uncertainty about the equation's structure, with a learned reliability for each expert and topic, should recover correct equations at noise levels where autonomous methods fail, using far fewer questions than eliciting expert opinion on every candidate term. The claim is falsifiable. If uncertainty from a single formulation selects questions just as well, the formulation-mixture ingredient is unnecessary. If a one-shot full elicitation matches the result at equal cost, the active selection is unnecessary.
Where This Sits in Prior Work
Several papers motivate the gap. The Novelty Ceiling proves that automated evaluators in discovery loops become confidently wrong outside their training regime. It shows that routing questions to humans by structural novelty beats uncertainty-based routing on the Feynman equations (21.1% vs 15.8% discovery rate at equal budget). However, it treats humans as infallible oracles and works on noise-free symbolic regression, not on differential equations recovered from noisy trajectories. SRBench++ brings domain experts into symbolic-regression evaluation and finds two things: high noise makes algorithms invent spurious complexity, and expert trust tracks causal plausibility rather than $R^2$ or expression length. Its experts score finished models after the fact and never steer the search. We put them inside the loop under a budget.
Other papers supply methods we reuse. Ensemble-SINDy bags data and libraries and uses weak forms to obtain term inclusion probabilities, tolerating up to 100% noise in some PDE cases. Its "active learning" chooses new trajectories, not human questions. UQ-SINDy gives spike-and-slab posteriors (a prior that puts each term either exactly at zero or at a nonzero value) over term inclusion, which is the natural place to inject expert evidence. IDENT contributes the forward-simulation error that discards spurious PDE terms. Kaheman et al. contributes joint denoising, which we use as one of the formulations. ATSBL, like E-SINDy, does active sampling by physically perturbing the system with no human involvement.
From the annotation side, PU-ADKA shows that modeling each expert's knowledge boundary and cost matters under a fixed monetary budget. On its text QA benchmark, boundary-aware allocation reaches an 18.2% win rate against 4.7% for random allocation. We borrow the idea of per-expert, per-topic competence, not its reinforcement-learning machinery. Two papers constrain the design. DIPS and Interpreting Expert Annotation Differences in Animal Behavior establish that expert labels are noisy and that feeding noisy labels back into a learner causes compounding errors. For this reason expert answers enter as likelihoods with estimated reliability, never as hard constraints.
Research Program
The method has three parts.
The first part is a formulation-mixture posterior over structure. Let $z \in \{0,1\}^p$ indicate which of $p$ library terms are present. For each formulation $f$ (strong-form with smoothed derivatives, weak form, jointly denoised regression, and forward-simulation-scored regression), we compute a spike-and-slab posterior $p_f(z \mid D)$ from data $D$, following UQ-SINDy. To keep cost low, we approximate each posterior with bootstrap ensembles in the E-SINDy style rather than full MCMC. Expert answers $A$ then enter as
$$ p(z \mid D, A) \propto \Big[\sum_f w_f\, p_f(z \mid D)\Big] \prod_{q} \rho_{e_q, c_q}^{\,\mathbb{1}[a_q = s_q(z)]}\,(1-\rho_{e_q, c_q})^{\,\mathbb{1}[a_q \neq s_q(z)]}. $$
Here $w_f$ are formulation weights, $q$ indexes questions already asked, $s_q(z) \in \{0,1\}$ is the truth value of question $q$'s statement under structure $z$, $a_q$ is the answer received from expert $e_q$, and $\rho_{e,c}$ is that expert's probability of answering correctly on topic class $c$. The weights $w_f$ are set by held-out forward-simulation error. The reliabilities $\rho$ get a Beta prior and are updated jointly, in the spirit of classic annotator-agreement models, whenever the data later resolve a question with high confidence.
The second part is the design of the questions themselves. Experts are not asked about individual monomials, which they answer unreliably. Each question is a mechanistic statement that maps to a set of library terms. Examples are "does species $x$ inhibit the growth of $y$?", "is total mass conserved?", "is there diffusion in this variable?", and "is the response saturating?". Each statement $s_q(z)$ is a logical function of $z$, so one answer can move many terms at once. This also makes piDMD-style structural constraints from Baddoo et al. expressible as questions instead of hard-coded assumptions.
The third part is budgeted selection. The next question and expert pair maximizes expected entropy reduction in $p(z \mid D, A)$ per unit cost, weighted by how much the answer would change the model's predicted trajectories outside the training window. The weighting exists because SRBench++ reports that extrapolation is where noisy discovered equations fail. The loop stops when the budget is spent or when the best structure's posterior mass exceeds a threshold.
The analysis component asks one question: when does formulation disagreement correctly flag spurious terms? We will measure, across noise levels, the fraction of falsely included terms that have high single-formulation confidence but high mixture entropy. That fraction is the mechanism's direct test.
Baselines
The naive reference point is autonomous discovery with no expert at all. We use E-SINDy with weak forms, the strongest noise-robust sparse method in the file. It is run on identical trajectories, with its library and thresholds tuned on a separate validation system using the same tuning budget our method gets. As a reported anchor, E-SINDy handles "up to 100% noise levels in specific cases" on Burgers. R-DISCOVER reports 2.28% coefficient error on Burgers at up to 125% white noise, and we include it as the strongest open-form autonomous competitor on that PDE. We found no reported result on Lacuna for exact support-recovery rates of these methods on Lorenz or Lotka–Volterra at matched noise levels, so we will measure them ourselves. Beating autonomy means higher exact structure recovery and lower extrapolation error at the same noise level, using at most a few dozen questions.
The strongest human-in-the-loop competitor is full up-front elicitation. Every expert answers every mechanistic question once, the answers enter the same posterior, and the cost is the full question set. Beating it means matching its recovery rate at a small fraction of its question cost. If it wins at equal cost, we report that honestly.
The second interactive baseline is single-formulation uncertainty sampling. We use the same question space and the same expert-reliability model, but questions are chosen by the entropy of the E-SINDy inclusion probabilities alone. This is the uncertainty-based routing that The Novelty Ceiling argues against, transplanted to equation discovery. It doubles as the key ablation, because it differs from our method only in removing the formulation mixture. A further ablation removes reliability modeling ($\rho = 1$, experts treated as oracles) to test whether the DIPS-style concern about noisy labels matters here. Novelty-based routing from The Novelty Ceiling, which uses tree edit distance to previously validated structures, is included as a third selection rule. Random question selection serves as the floor.
All interactive methods share the same simulated experts, the same budget of 10, 25, and 50 questions, and the same random seeds.
Why not just…?
Why not just denoise harder and skip the experts? Better smoothing does help, and our method uses joint denoising as one formulation. But SRBench++ reports that state-of-the-art algorithms still select noise-fitting terms at high noise, and Fallah Mortezanejad et al. found that direct filtering followed by standard regression failed on real multivariate series. The autonomous baseline arm measures this directly. If E-SINDy with the best denoiser already recovers the structure at a given noise level, we report that experts add nothing there and restrict the claim to the noise levels where it fails.
Why not just ask the expert to approve the whole library once? For a small polynomial library this is a genuine threat, since a 3-variable degree-2 system has only a few dozen mechanistic questions. Our bet is that the advantage grows with library size (PDE libraries with many derivative interactions) and with expert fallibility, because active selection can ask a reliable expert again about the contested terms instead of trusting one pass. The full-elicitation baseline at matched cost decides this. We will report the library size at which, if ever, the curves cross.
Why not just query where E-SINDy's inclusion probability is near one half? This is the most natural competitor. Its failure mode is that spurious terms under noise are often included confidently, so they never look uncertain. That is the equation-discovery analogue of the confidently-wrong evaluators in The Novelty Ceiling. The single-formulation ablation isolates exactly this. If it performs as well as the formulation mixture, the central mechanism is refuted.
Why not just ask a large language model instead of a human? An LLM can answer textbook mechanism questions cheaply, so we include it as an additional low-cost "expert" with its own estimated reliability rather than ruling it out. The Novelty Ceiling argues that automated evaluators cannot judge structures far from their training distribution, and PU-ADKA motivates human acquisition precisely because LLMs lack recent specialized knowledge. Our real-data case of actin aggregation under different buffers tests whether the LLM's reliability collapses on non-textbook mechanisms.
Proposed Experiments
The synthetic systems are Lotka–Volterra, Lorenz, Burgers, and Kuramoto–Sivashinsky, at noise levels from 1% to 100% of signal standard deviation. On these, simulated experts have known topic-specific reliabilities between 0.6 and 0.95. Their knowledge is bounded in the PU-ADKA style: one expert is reliable on reaction terms and near chance on transport terms, and another is the reverse. A fraction of their wrong answers are confident. The metrics are exact structure recovery rate, coefficient error on recovered terms, extrapolation error over a window twice the training length, and questions needed to reach 80% recovery.
The real-data cases are the 21-point Hudson Bay lynx–hare series used by E-SINDy and UQ-SINDy, the actin aggregation data used by ODENet, and the SRBench++ New York COVID-19 track. On these we recruit two or three real domain experts (an ecologist, a biophysicist, an epidemiologist) under a fixed time budget. Here ground truth is unknown. Instead, we score held-out forecast error and blinded expert plausibility ratings by a separate expert, mirroring the trust scores in SRBench++.
The decisive checks are three. First, at noise levels where autonomous E-SINDy fails, the formulation-mixture selector must beat single-formulation uncertainty sampling at equal budget; otherwise the core claim falls. Second, our method must reach full-elicitation accuracy at substantially lower question cost on the PDE libraries. Third, with reliability modeling turned off, recovery must degrade as simulated expert error rises, confirming that treating experts as oracles is the wrong assumption.
References
- The Novelty Ceiling: PAC-Theoretic Bounds on Autonomous Scientific Discovery and the Minimum Oversight Rate
- SRBench++: Principled Benchmarking of Symbolic Regression With Domain-Expert Interpretation
- Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control
- Sparsifying priors for Bayesian uncertainty quantification in model discovery
- IDENT: Identifying Differential Equations with Numerical Time Evolution
- Automatic differentiation to simultaneously identify nonlinear dynamics and extract noise probability distributions from data
- Sparse inference and active learning of stochastic differential equations from data
- Active Domain Knowledge Acquisition with 100-Dollar Budget: Enhancing LLMs via Cost-Efficient, Expert-Involved Interaction in Sensitive Domains
- You can't handle the (dirty) truth: Data-centric insights improve pseudo-labeling
- Interpreting Expert Annotation Differences in Animal Behavior
- Physics-informed dynamic mode decomposition
- Revealing hidden dynamics from time-series data by ODENet