Introduction
Large Language Models (LLMs) have demonstrated remarkable capabilities in synthesizing existing knowledge, yet they often struggle with genuine scientific innovation. When tasked with generating research ideas, these models frequently encounter a "diversity wall," resulting in repetitive or incremental suggestions. This limitation stems from a lack of goal-oriented external knowledge acquisition; standard Retrieval-Augmented Generation (RAG) typically retrieves similar content rather than the "distant" knowledge required for creative breakthroughs.
Methodology: The Nova Pipeline
To address these challenges, we introduce Nova, a framework designed to proactively identify and retrieve external knowledge to fill reasoning gaps. The Nova pipeline consists of three distinct stages:
- Initial Seed Idea Generation: Utilizing a "seed paper" and 10 fundamental scientific discovery theories (e.g., Kuhn’s paradigm shifts), the system prompts the LLM to adopt the perspective of a philosopher of science to generate initial concepts.
- Iterative Planning and Search: This core stage involves multiple iterations where the LLM devises a search plan to identify specific fields or papers that could enhance novelty. It then retrieves these papers via the Semantic Scholar API and uses self-correction to refine the idea pool.
- Idea Completion: Promising seed ideas are expanded into comprehensive research proposals, including methods and experimental plans.
Experimental Evaluation
Nova was evaluated using a dataset of 170 high-quality papers from top-tier 2024 conferences (CVPR, ACL, ICLR). It was compared against state-of-the-art (SOTA) baselines, including AI-Researcher, AI-Scientist, and Research-Agent.
Key Results
The effectiveness of Nova's iterative approach is evident in both automated and human evaluations:
- Novelty and Diversity: Nova produced 3.4 times more unique novel ideas than baselines without iterative planning. As shown in the performance data, the number of unique novel ideas increases significantly with each iteration step.
- Tournament Performance: In a "Swiss Tournament" head-to-head ranking, Nova generated 2.5 times more top-rated ideas (619) compared to the next best method, AI-Scientist (249).
- Human Validation: PhD-level experts confirmed that Nova contributed 37.5% of the "Top 4" ideas in comparative trials, while maintaining the lowest frequency of poor-quality ideas.
Discussion and Conclusion
The success of Nova highlights that the strategy of search is more critical than the volume of search. By forcing the model to plan its information needs before retrieval, Nova avoids the echo chamber of its training data. While gains tend to plateau after three iterations, future work could involve formal reward functions or human-in-the-loop "co-pilot" configurations to further push the boundaries of AI-driven scientific discovery.
Summary updated