TL;DR
AI scientists often struggle with "incrementalism," proposing obvious solutions because they are anchored by too much task-specific information. This paper introduces BVSR (Blind Variation, Selective Retention), a framework that deliberately limits task details during ideation to foster "out-of-the-box" cross-field creativity, then uses a learned selector to filter the best candidates for expensive empirical testing.
What Problem Does This Paper Address?
While Large Language Models (LLMs) have shown promise in automating scientific discovery, they tend to produce "safe" or familiar ideas. The authors identify a core culprit: information-induced anchoring. When an AI is given a highly detailed task description (e.g., "Find spatially variable genes in mouse eye transcriptomics"), it tends to retrieve standard domain-specific methods.
The challenge is to move AI scientists beyond incremental modifications toward truly creative discoveries—ideas that are simultaneously original (rarely proposed), useful (perform well), and non-obvious (not predictable from existing literature).
Methods
Inspired by Donald Campbell’s 1960 cognitive theory, the authors propose a two-stage framework:
- Blind Variation (BV): During the ideation phase, the system provides the LLM with a "limited context" summary of the task. By stripping away domain-specific jargon and specific data labels, the AI is forced to think in terms of abstract mechanisms. This encourages it to "borrow" successful concepts from distant fields (e.g., applying physics principles to biological data).
- Selective Retention (SR): Because "blind" ideation produces a high volume of diverse ideas—some of which are inevitably duds—the system needs a filter. The authors train a learned SR selector using a pairwise logistic loss on previously verified proposals. This selector learns to identify which ideas are worth the "costly" step of implementation and empirical verification.
The authors also formalize a quantitative metric for creativity: $$Creativity = Originality \times Utility \times Non\text{-}obviousness$$
Datasets
The framework was evaluated across 12 scientific tasks in four domains:
- Biology: Single-cell analysis tasks like spatially variable gene detection and cell-cell communication.
- Machine Learning: Unsupervised anomaly detection and discovering scaling laws for model training.
- Astrodynamics: Trajectory design for the Cassini and Voyager 2 missions.
- Neuroscience: Whole-brain neural activity forecasting (ZAPBench).
The researchers tested five mainstream LLMs as "proposers," including Kimi-K3, DeepSeek-V4, and GLM-5.
Key Insights and Ideas
- The Information-Exploration Tradeoff: There is a measurable decline in originality as task information increases. Providing just ~10 bits of information results in significantly more creative top-tier proposals than providing 100+ bits.
- Cross-Field "Transplants": By limiting context, the AI discovered that the Nudged Elastic Band method (from transition-state physics) could outperform standard biological baselines for gene expression analysis.
- Mathematics as a Creative Engine: When analyzing "Domains of Inspiration," the authors found that papers in Mathematics had the highest likelihood of inspiring "top 1%" creative proposals, while Biology/Medicine led in utility.
- Learned vs. Zero-Shot Selection: Standard LLM "judges" are surprisingly poor at predicting which ideas will actually work. A selector trained on empirical outcomes (SR) is far more effective at discarding the bottom 50% of ideas while retaining nearly all top-performing ones.
Key Results
- Higher Creativity: BV (Limited Information) achieved a creativity score of 0.530, significantly outperforming BV with Full Information (0.462) and other prompting baselines like analogical reasoning.
- Efficiency Gains: When integrated into the
SimpleTESworkflow, BVSR reached the same discovery performance as the baseline while requiring 1.6x to 3.8x fewer verification steps. - SOTA Discoveries: In anomaly detection, BVSR achieved a score of 0.679, crushing the best AI scientist baseline (0.629) and the standard Isolation Forest baseline (0.491).
Limitations and Future Work
- Inference Costs: While BVSR reduces the number of verifications (which are often the most expensive part of the pipeline), it requires generating a larger pool of initial ideas, which increases LLM inference costs.
- LLM as Judge: While validated by PhD-level experts, the metrics for "originality" and "non-obviousness" still rely on LLM-based semantic grouping and judging.
- Future Directions: The authors suggest exploring how to dynamically adjust the "information level" based on the task's difficulty and investigating more complex "selective retention" models that can provide feedback to the ideation stage.
Summary updated