Introduction
Traditional AI-driven data collection frameworks, such as active learning or reinforcement learning, typically operate under pre-defined objective functions. While efficient for known goals, this approach often fails to capture the serendipitous nature of scientific discovery. Historical breakthroughs, like the discovery of penicillin, frequently stem from researchers noticing unexpected anomalies that fall outside original experimental designs.
To bridge this gap, we present a framework for human-AI collaboration that allows scientists to identify interesting patterns as they emerge. The core of this framework is the Threshold Estimating Sampling Algorithm (TESA), a zero-regret algorithm designed to prioritize sampling in scientifically promising areas based on iterative human feedback.
Methodology
TESA facilitates a dynamic interaction where a human researcher periodically provides feedback by marking "keypoints"—specific data regions that appear scientifically significant. This addresses the fundamental explore-exploit dilemma: determining whether the AI should gather more data at a known keypoint to increase statistical confidence or explore seemingly uninteresting areas for new discoveries.
Technical Innovations
- Bayesian Thresholding: TESA models the human decision-making process by assuming a threshold ($h$) of samples ($n_i$) is required for a point to be deemed interesting. Using a Pareto distribution to estimate this threshold, the algorithm can effectively prune the search space by stopping sampling in areas that exceed the threshold without being marked as keypoints.
- Zero-Regret Guarantee: The framework introduces a novel definition of "external regret" for human-in-the-loop settings. TESA is mathematically proven to achieve zero regret asymptotically, ensuring long-term optimal performance.
- Iterative Feedback Loop: Unlike static models like Uncertainty Weighted Posterior Sampling (UWPS), TESA allows humans to add or modify keypoints every $I$ timesteps, accommodating the evolving intuition of a researcher as data is collected.
Experimental Evaluation
TESA was evaluated using "replay" simulations across three real-world scientific datasets: Economics (avocado price vs. sales), Mental Health (tech worker demographics vs. diagnosis rates), and Cognitive Psychology (stimulus size vs. reaction time). Additionally, a human subject experiment involving 101 participants was conducted to compare TESA against traditional workflows requiring upfront "prior theories."
Results and Discussion
In all simulated scientific domains, TESA variants consistently outperformed standard baselines, including Round Robin, Epsilon-Greedy, and Thompson Sampling. Notably, TESA significantly surpassed UWPS, which relies on static prior predictions rather than dynamic feedback.
Key Insights
- Dynamic Interest: Scientific interest is non-stationary. TESA effectively manages this by allowing for the re-evaluation of data points as more information becomes available.
- Modeling the Human: By estimating the "human threshold" for interest, TESA prunes the search space more effectively than algorithms that only model the data itself.
- User Preference: In the human subject study, 95% of participants who passed comprehension checks preferred TESA's iterative data collection over the prior-theory method, citing that TESA-collected data was more "scientifically interesting."
Conclusion and Future Work
While TESA shows great promise for 1D response functions, future work will address higher-dimensional spaces and physical constraints, such as the movement costs of robotic sampling systems. Additionally, while asymptotic zero-regret is proven, establishing specific convergence rates remains a priority for theoretical advancement. TESA represents a significant step toward AI systems that truly augment human scientific intuition rather than just automating pre-defined tasks.
Summary updated