Abstract
Test-time scaling has emerged as a critical frontier for improving the performance of Large Language Models (LLMs) on complex reasoning tasks. However, existing methods often struggle with either high variance in sequential self-correction or a lack of iterative refinement in parallel sampling. This paper introduces Population-Evolve, a training-free evolutionary method that maintains and iteratively refines a parallel population of candidate solutions. By leveraging the comparative capabilities of LLMs within a Genetic Algorithm (GA) framework, Population-Evolve achieves superior accuracy and stability. Experimental results on competition-level benchmarks (HMMT, AIME, BRUMO) demonstrate that Population-Evolve outperforms standard majority voting and sequential baselines while reducing runtime by over 74% compared to state-of-the-art iterative methods.
1. Introduction
As LLMs are applied to increasingly sophisticated mathematical challenges, such as those found in the American Invitational Mathematics Examination (AIME) or the International Mathematical Olympiad (IMO), standard zero-shot prompting often falls short. Researchers have turned to "test-time scaling"—allocating additional computational resources during inference to allow the model to verify, critique, and refine its outputs.
While effective, current training-free test-time strategies face a trade-off between depth and stability. Sequential methods, which refine a single answer over multiple steps, are prone to "confirmation bias" and high performance variance. Conversely, parallel sampling methods like Majority Voting are stable but fail to improve the underlying quality of the generated candidates. Population-Evolve bridges this gap by applying evolutionary principles to a population of solutions, enabling parallelized, iterative refinement.
2. The Limitations of Current Test-Time Scaling
Existing approaches to test-time scaling generally fall into two categories, both of which possess notable flaws:
- Sequential Evolving (e.g., DSER): These methods involve a model iteratively critiquing and refining a single solution trajectory. Because the process is serial, it is computationally slow. Furthermore, it is highly unstable; a single erroneous critique can derail the entire reasoning chain, leading to significant fluctuations in accuracy across different runs.
- Selection After Sampling (e.g., GenSelect): These methods generate a large batch of answers in parallel and use a reward model or the LLM itself to select the best candidate. While more stable than sequential methods, they are limited by the quality of the initial sample; the model acts only as a judge, not as an editor capable of synthesizing better logic.
3. Methodology: The Population-Evolve Framework
Population-Evolve treats the reasoning process as a Genetic Algorithm defined by population size ($P$), iteration count, and evolution operators. The process consists of three primary phases:
3.1 Initialization (Generation 0)
The LLM generates an initial population of $P$ independent candidate solutions using standard Chain-of-Thought (CoT) prompting. This provides a diverse baseline of reasoning paths.
3.2 Iterative Evolution
In each subsequent generation, the model is prompted to produce a new solution. Unlike standard self-correction, the prompt for each parallel thread includes the original problem statement and the entire population of solutions from the previous generation. This allows the model to perform comparative verification—identifying contradictions across different candidates and synthesizing the most robust logical steps.
To maintain efficiency and prevent context window overflow, the authors implement a context optimization strategy: they strip the lengthy <think>...</think> reasoning tokens from the historical population, retaining only the final detailed solution steps. This ensures the model receives the core logic of its peers without being overwhelmed by redundant internal monologues.
3.3 Selection and Convergence
After a fixed number of iterations (typically 4 to 8), or once the population reaches a consensus, a standard majority voting mechanism is applied to the final generation to determine the final answer. This population-based approach ensures that the final output is the product of multiple refined trajectories rather than a single potentially flawed one.
4. Experimental Evaluation
4.1 Datasets and Models
The method was evaluated on three rigorous, competition-level math benchmarks:
- HMMT24-25: 60 problems from the 2024 and 2025 MathArena sets.
- AIME25: 30 problems from the American Invitational Mathematics Examination.
- BRUMO25: 30 high-difficulty mathematical problems.
The authors utilized three open-source reasoning models: Qwen3-4B-Thinking, DeepSeek-R1-Distill-Qwen3-8B, and OpenReasoning-Nemotron-7B.
4.2 Results
- Accuracy Gains: Population-Evolve consistently outperformed strong baselines, including Majority Voting @ 64 and DSER. On the HMMT24-25 benchmark, it achieved an average accuracy improvement of +7.92% over Majority Voting.
- Efficiency: By parallelizing the refinement process, Population-Evolve achieved a 74.13% reduction in runtime compared to the sequential DSER method.
- Model Performance: Enhanced 7B and 8B models using Population-Evolve outperformed the zero-shot performance of significantly larger proprietary models, such as Gemini-1.5-Pro and GPT-4o, on the tested benchmarks.
5. Discussion and Insights
The success of Population-Evolve highlights two key insights into LLM behavior. First, LLMs are significantly better at comparative verification than absolute verification; they struggle to find errors in a vacuum but excel at spotting them when presented with conflicting alternatives. Second, population convergence acts as a stabilizer. While single-trajectory methods are volatile, the collective evolution of a population naturally filters out outliers and converges on mathematically sound logic.
6. Conclusion
Population-Evolve provides a robust, efficient framework for test-time scaling in mathematical reasoning. By combining the strengths of parallel sampling with the depth of iterative refinement, it allows smaller open-source models to compete with proprietary giants. Future research will explore training models specifically for this evolutionary paradigm and implementing real-time communication between parallel generation threads to further accelerate logical convergence.
Summary updated