Abstract
Large language models (LLMs) are increasingly being used to generate synthetic datasets for the evaluation and training of downstream models. However, prior work has noted that such generated data lacks diversity. In this paper, we propose Voyager, a novel principled approach to generate diverse datasets. Our approach is iterative and directly optimizes a mathematical quantity that optimizes the diversity of the dataset using the machinery of determinantal point processes. Furthermore, our approach is training-free, applicable to closed-source models, and scalable. In addition to providing theoretical justification for the working of our method, we also demonstrate through comprehensive experiments that Voyager significantly outperforms popular baseline approaches by providing a 1.5-3x improvement in diversity.
Introduction
Large Language Models (LLMs) have become a cornerstone for generating synthetic data used in training and evaluating other models. However, a significant hurdle remains: the "diversity deficit." LLMs often produce repetitive content or collapse into a few common semantic clusters, even when sampling temperatures are high. Existing solutions, such as top-p sampling or prompt-based requests for "different topics," often fail to scale or require extensive domain expertise. Training-based methods are computationally expensive and restricted to open-weight models.
VOYAGER addresses these challenges as a training-free framework that leverages Determinantal Point Processes (DPP) to generate highly diverse synthetic datasets. By treating diversity as a geometric "volume" and using natural language feedback to steer prompts away from redundant areas, VOYAGER achieves a 1.5x–3x improvement in diversity over standard methods without sacrificing data quality.
The VOYAGER Framework
VOYAGER conceptualizes dataset generation as an iterative exploration of a "data manifold." The framework consists of three primary components:
- Diversity as Volume: Utilizing Determinantal Point Processes (DPP), VOYAGER represents data points as vectors. The diversity of a set is proportional to the volume of the parallelepiped spanned by these vectors. A higher determinant in the similarity matrix indicates a more spread-out, less redundant dataset.
- The Anchor Set: To maintain computational efficiency, VOYAGER maintains a small, representative "anchor set" of salient data points. New data is only accepted if it provides a significant "marginal gain" in volume relative to this anchor set, avoiding the need for exhaustive pairwise comparisons.
- Textual Gradients: When the LLM generates redundant data, VOYAGER prompts the model to critique its own output. It identifies the cause of redundancy and generates "textual gradients"—natural language instructions—to refine the prompt. This steers future "explorers" (prompts) toward unexplored regions of the data space.
Experimental Evaluation
The researchers evaluated VOYAGER across diverse tasks, including creative writing (topical sentences, poems, movie plots) and reasoning (grade school math, logical puzzles).
Key Results
- Significant Diversity Gains: VOYAGER outperformed all baselines, including hierarchical and history-conditioned prompting. It demonstrated an average 296% improvement in Vendi Score for creative tasks and 412% for reasoning tasks.
- Quality Preservation: LLM-as-a-judge evaluations confirmed that the increase in diversity did not compromise data quality; in some cases, quality even improved.
- Downstream Performance: A Gemma-7B model trained on 1,000 VOYAGER-generated math problems achieved 45.7% accuracy, significantly higher than the 35.7% achieved with standard LLM data.
Discussion and Conclusion
VOYAGER bridges abstract geometry and practical curation by providing a formal proof that the "effective rank" of a similarity matrix can be approximated by its determinant. The "explorer" analogy—where prompts are agents mapping a territory—provides an intuitive grasp of the algorithm's function. By using textual gradients, VOYAGER achieves a form of optimization in natural language space, making it compatible with any API-based model.
While currently limited to text and dependent on automated similarity kernels, VOYAGER represents a major step forward in data efficiency. The finding that 500 diverse samples can be as effective as 1,000 standard samples suggests that diversity is a powerful lever for reducing the costs associated with training high-performance models.
Summary updated