Adaptive Experimentation Can Hide Model Error:Detection Limits and Falsification Budget

Ramnarayan ChoudharyarXiv 2026 · arXiv

TL;DR

Adaptive experimental design, while efficient at refining model parameters, can inadvertently "hide" fundamental model errors by repeatedly sampling only where the incorrect model happens to fit. This paper introduces a formal "power ceiling" theorem that quantifies this blindness and proposes a practical, training-free "falsification budget" rule to ensure scientists don't miss the forest for the trees.

What Problem Does This Paper Address?

In modern scientific discovery, we often use adaptive algorithms to choose the next experiment. A standard objective is Expected Information Gain (EIG), which picks measurements that most reduce uncertainty about a model's parameters.

The catch? This assumes your model is actually correct. If your model is fundamentally misspecified (e.g., you're fitting a straight line to data that has a hidden "bump"), an EIG-based optimizer might keep sampling at the endpoints where the line fits perfectly. By doing so, it never sees the "bump" and remains blissfully unaware that its model is wrong. The authors call this an information bottleneck created by the experimenter's own sampling policy.

Methods

The paper approaches this problem through both formal theory and algorithmic intervention:

  1. Own-Design Power Ceiling (Theorem 1): The authors prove a fundamental limit on error detection. If a sampling policy concentrates on a design set where at least one member of the working model class agrees with the truth, the probability of ever detecting the model error is bounded by the significance level $\alpha$. Essentially, you can't reason your way out of data you didn't collect.
  2. The Blindness Certificate ($B_t$): They derive a computable diagnostic that splits into two parts: Ignorance ($U_t$), which measures disagreement among hypotheses in a model expansion, and Evidence ($D_t$), which measures their average disagreement with the current working model.
  3. Falsification Budget (Algorithm 1): They propose a "Gate + floor" rule. When the blindness certificate exceeds a threshold (the "gate"), the algorithm stops optimizing for parameters and instead picks the design that most distinguishes the working model from a broader expansion. A "floor" of random exploration ensures the system never stops probing for errors entirely.
  4. Anytime-Valid Testing: To maintain statistical rigor under adaptive sampling, they use e-processes and Ville’s inequality, allowing for model rejection at any point in time without "p-hacking" concerns.

Datasets

The authors evaluate their methods across a diverse range of environments:

  • Generic Design Families: Seven grid-exact families including dose-response (logistic), linear regression, and Poisson rates.
  • Mechanistic Environments: Physically realistic simulations of enzyme kinetics (Michaelis-Menten) and pharmacokinetics.
  • Measured-Replicate Replay: A "corrected replay" using real-world experimental data from drug-combination screens (e.g., Jaaks et al., 2022), where the algorithm chooses from actual recorded replicate outcomes.
  • LLM Experimenters: A novel benchmark where frontier Large Language Models (like GPT-6 and Gemini 4) act as the experimental designers.

Key Insights and Ideas

  • Capability Can Cause Blindness: Perhaps the most striking finding is that "smarter" experimenters can be worse at detecting errors. As shown in Figure 2, frontier reasoning models (red dots) concentrate their designs so aggressively—much like a greedy EIG optimizer—that they lose the ability to detect planted model errors that simpler, "messier" models (blue dots) catch easily.
Frontier models concentrate designs and lose detection power compared to random sampling.
  • Joint Coverage is King: Detecting a model error isn't just about visiting an "unusual" setting; it often requires a specific combination of settings that collectively rule out all members of a model class.
  • The Detection-Accuracy Trade-off: Reserving a budget for falsification naturally incurs a small "premium" in parameter prediction risk. However, the authors show this cost is often negligible compared to the massive gain in error detection.

Key Results

The empirical record demonstrates that standard adaptive designs are frequently "blind" to misspecification, but the proposed falsification rule can recover significant detection power.

  • Detection Recovery: In several settings where greedy EIG had near-zero detection, the "Gate + floor" rule recovered substantial power. For example, in a fresh continuous-null panel ($T=240$), the revised rule detected 35.8% of misspecified campaigns compared to just 2.78% for the "frozen" baseline.
  • LLM Performance: While base LLMs often failed to detect errors due to design concentration, adding a "blindness monitor" to the prompt or using the falsification harness restored detection rates to levels at or above random sampling (see Figure 2a).
  • Heterogeneity: As seen in Figure 1, the effectiveness of the intervention varies across settings, highlighting that no single design policy is universally superior.
Detection recovery varies across different experimental settings and comparators.

Limitations and Future Work

  • Expansion Dependence: The falsification rule relies on a declared model expansion ($M_1$). It can help you find errors you've considered (e.g., "maybe there's a bump"), but it won't necessarily find "unknown unknowns" outside that expansion.
  • Setting Sensitivity: The "gate" constant ($b^*$) required recalibration for longer horizons, suggesting that a truly universal, self-tuning threshold remains an open challenge.
  • Real-world Validation: While "measured replays" are a step forward, the authors emphasize that prospective lab experiments are needed to establish these methods for actual scientific discovery.
  • Computational Overhead: The continuous-null version of the detector involves significant numerical optimization, which might be a bottleneck for high-speed automated science.

Summary updated

Abstract

An experiment can be informative about the parameters of a working model and uninformative about whether that model is wrong. Adaptive design can reinforce this mismatch by repeatedly selecting settings where an incorrect model fits. We bound the power of any valid adequacy test in terms of the experimenter's own sampling distribution, then distinguish information absent from the selected data from information lost by the detector. A computable diagnostic measures unresolved error relative to a declared expansion of the working model. It motivates a training-free rule that reserves part of the design budget for falsification. The empirical record is mixed: the independently specified primary study fails detection eligibility, and a later numerator diagnostic fails its improvement criteria. After the frozen rule fails on fresh continuous-null panels, a disclosed secondary horizon-calibrated revision detects 35.8% of misspecified campaigns versus 31.7% for a class-separation control at T = 240, but has no resolved advantage at T = 480. In a corrected 289-unit measured replay it improves on random design and trails class-discrimination EIG. A conditional futility bound explains why remaining time matters without proving the gate constant or its power. These results support separating parameter learning, model checking and detector performance; they do not establish a universally superior design policy or prospective scientific discovery.
BibTeX
@misc{choudhary2026adaptive,
  author = {Ramnarayan Choudhary},
  title = {{Adaptive Experimentation Can Hide Model Error:Detection Limits and Falsification Budget}},
  year = {2026},
  eprint = {2611.07529},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.07529},
}
Figure