Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning

Yanzhi Zhang, Yitong Duan, Zhaoxi Zhang, Jiyan He, Shuxin ZhengarXiv 2025 · DOI · arXiv

Abstract

Test-time scaling has emerged as a critical frontier for improving the performance of Large Language Models (LLMs) on complex reasoning tasks. However, existing methods often struggle with either high variance in sequential self-correction or a lack of iterative refinement in parallel sampling. This paper introduces Population-Evolve, a training-free evolutionary method that maintains and iteratively refines a parallel population of candidate solutions. By leveraging the comparative capabilities of LLMs within a Genetic Algorithm (GA) framework, Population-Evolve achieves superior accuracy and stability. Experimental results on competition-level benchmarks (HMMT, AIME, BRUMO) demonstrate that Population-Evolve outperforms standard majority voting and sequential baselines while reducing runtime by over 74% compared to state-of-the-art iterative methods.

1. Introduction

As LLMs are applied to increasingly sophisticated mathematical challenges, such as those found in the American Invitational Mathematics Examination (AIME) or the International Mathematical Olympiad (IMO), standard zero-shot prompting often falls short. Researchers have turned to "test-time scaling"—allocating additional computational resources during inference to allow the model to verify, critique, and refine its outputs.

While effective, current training-free test-time strategies face a trade-off between depth and stability. Sequential methods, which refine a single answer over multiple steps, are prone to "confirmation bias" and high performance variance. Conversely, parallel sampling methods like Majority Voting are stable but fail to improve the underlying quality of the generated candidates. Population-Evolve bridges this gap by applying evolutionary principles to a population of solutions, enabling parallelized, iterative refinement.

2. The Limitations of Current Test-Time Scaling

Existing approaches to test-time scaling generally fall into two categories, both of which possess notable flaws:

  1. Sequential Evolving (e.g., DSER): These methods involve a model iteratively critiquing and refining a single solution trajectory. Because the process is serial, it is computationally slow. Furthermore, it is highly unstable; a single erroneous critique can derail the entire reasoning chain, leading to significant fluctuations in accuracy across different runs.
  2. Selection After Sampling (e.g., GenSelect): These methods generate a large batch of answers in parallel and use a reward model or the LLM itself to select the best candidate. While more stable than sequential methods, they are limited by the quality of the initial sample; the model acts only as a judge, not as an editor capable of synthesizing better logic.

3. Methodology: The Population-Evolve Framework

Population-Evolve treats the reasoning process as a Genetic Algorithm defined by population size ($P$), iteration count, and evolution operators. The process consists of three primary phases:

3.1 Initialization (Generation 0)

The LLM generates an initial population of $P$ independent candidate solutions using standard Chain-of-Thought (CoT) prompting. This provides a diverse baseline of reasoning paths.

3.2 Iterative Evolution

In each subsequent generation, the model is prompted to produce a new solution. Unlike standard self-correction, the prompt for each parallel thread includes the original problem statement and the entire population of solutions from the previous generation. This allows the model to perform comparative verification—identifying contradictions across different candidates and synthesizing the most robust logical steps.

To maintain efficiency and prevent context window overflow, the authors implement a context optimization strategy: they strip the lengthy <think>...</think> reasoning tokens from the historical population, retaining only the final detailed solution steps. This ensures the model receives the core logic of its peers without being overwhelmed by redundant internal monologues.

3.3 Selection and Convergence

After a fixed number of iterations (typically 4 to 8), or once the population reaches a consensus, a standard majority voting mechanism is applied to the final generation to determine the final answer. This population-based approach ensures that the final output is the product of multiple refined trajectories rather than a single potentially flawed one.

4. Experimental Evaluation

4.1 Datasets and Models

The method was evaluated on three rigorous, competition-level math benchmarks:

  • HMMT24-25: 60 problems from the 2024 and 2025 MathArena sets.
  • AIME25: 30 problems from the American Invitational Mathematics Examination.
  • BRUMO25: 30 high-difficulty mathematical problems.

The authors utilized three open-source reasoning models: Qwen3-4B-Thinking, DeepSeek-R1-Distill-Qwen3-8B, and OpenReasoning-Nemotron-7B.

4.2 Results

  • Accuracy Gains: Population-Evolve consistently outperformed strong baselines, including Majority Voting @ 64 and DSER. On the HMMT24-25 benchmark, it achieved an average accuracy improvement of +7.92% over Majority Voting.
  • Efficiency: By parallelizing the refinement process, Population-Evolve achieved a 74.13% reduction in runtime compared to the sequential DSER method.
  • Model Performance: Enhanced 7B and 8B models using Population-Evolve outperformed the zero-shot performance of significantly larger proprietary models, such as Gemini-1.5-Pro and GPT-4o, on the tested benchmarks.

5. Discussion and Insights

The success of Population-Evolve highlights two key insights into LLM behavior. First, LLMs are significantly better at comparative verification than absolute verification; they struggle to find errors in a vacuum but excel at spotting them when presented with conflicting alternatives. Second, population convergence acts as a stabilizer. While single-trajectory methods are volatile, the collective evolution of a population naturally filters out outliers and converges on mathematically sound logic.

6. Conclusion

Population-Evolve provides a robust, efficient framework for test-time scaling in mathematical reasoning. By combining the strengths of parallel sampling with the depth of iterative refinement, it allows smaller open-source models to compete with proprietary giants. Future research will explore training models specifically for this evolutionary paradigm and implementing real-time communication between parallel generation threads to further accelerate logical convergence.

Summary updated

Abstract

Test-time scaling has emerged as a promising direction for enhancing the reasoning capabilities of Large Language Models in last few years. In this work, we propose Population-Evolve, a training-free method inspired by Genetic Algorithms to optimize LLM reasoning. Our approach maintains a dynamic population of candidate solutions for each problem via parallel reasoning. By incorporating an evolve prompt, the LLM self-evolves its population in all iterations. Upon convergence, the final answer is derived via majority voting. Furthermore, we establish a unification framework that interprets existing test-time scaling strategies through the lens of genetic algorithms. Empirical results demonstrate that Population-Evolve achieves superior accuracy with low performance variance and computational efficiency. Our findings highlight the potential of evolutionary strategies to unlock the reasoning power of LLMs during inference.
BibTeX
@misc{zhang2025population,
  author = {Yanzhi Zhang and Yitong Duan and Zhaoxi Zhang and Jiyan He and Shuxin Zheng},
  title = {{Population-Evolve: a Parallel Sampling and Evolutionary Method for LLM Math Reasoning}},
  year = {2025},
  eprint = {2512.19081},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2512.19081},
}
Figure