Batching the Reflective Optimization Loop: Parallel Proposals Make GEPA Faster and Better

Jialin Zhang, Lakshya A Agrawal, Shangyin Tan, Donghyun Lee, Lutfi Eren Erdogan, Dan Klein, Koushik Sen, Alex Dimakis, Matei ZahariaarXiv 2026 · arXiv

TL;DR

By batching multiple mutation proposals in parallel, the GEPA optimization loop for agentic AI systems can achieve a 3–4× wall-clock speedup while simultaneously improving held-out test scores. This approach transforms a traditionally sequential, resource-heavy bottleneck into a parallelizable process that generalizes better to new data.

What Problem Does This Paper Address?

Optimizing an "agentic" AI system—improving the prompts or code that guide an LLM—is a notoriously slow and expensive process. State-of-the-art optimizers like GEPA (Generalized Evolutionary Prompt Adaptation) use a "reflective" loop: the system proposes a mutation, evaluates it, and uses that feedback to inform the next step.

The problem is that this loop is inherently sequential. Even if you have a massive cluster of GPUs or a high-rate API tier, the optimizer typically waits for one candidate to be fully evaluated before proposing the next. This leaves parallel evaluation capacity idle and can result in optimization runs that take many hours. Furthermore, repeatedly "steering" an agent based on a single small validation set can lead to overfitting, where the agent performs great on the validation data but fails on the actual test set.

Methods

The authors introduce Parallel Proposals, a $P \times N$ sampling strategy that extends the open-source GEPA framework.

Instead of the standard sequential approach (sampling one parent and one mutation), the batched iteration works as follows:

  1. Sample $P$ parents from the current "Pareto frontier" (the set of best-performing candidates across different trade-offs).
  2. Generate $N$ mutations for each parent using different mini-batches of data.
  3. Evaluate all $P \cdot N$ candidates concurrently.
This figure compares standard best-candidate selection with the new parallel proposal strategy, showing how it explores a broader search tree simultaneously.

By dispatching these reflection requests and evaluations in one batch, the system pays the "iteration latency" once for multiple candidates. The authors also provide a runtime model to predict speedups based on the size of the worker pool and the probability of a mutation being successful.

Datasets

The researchers evaluated their approach on two distinct benchmarks:

  • LiveBench-Math: A challenging set of competition-level math problems (AMC, AIME, etc.). The goal was to optimize a solver prompt with a budget of 5,000 metric calls.
  • HoVer: A multi-hop claim verification task requiring the agent to retrieve specific Wikipedia pages. The goal was to optimize a retrieval agent with a budget of 3,000 metric calls.

Key Insights and Ideas

  • Parallelism as Regularization: One of the most surprising findings is that batching doesn't just make things faster; it makes them better. By committing to multiple proposals simultaneously before adapting to validation results, the optimizer is less likely to overfit to the specific quirks of the validation set.
  • The "Faster and Better" Frontier: In many optimization problems, there is a trade-off between speed and quality. Here, parallel proposals move the needle in both directions at once, as seen in the primary results plot.
  • Task-Dependent Scaling: The optimal way to split the "width" of a batch depends on the task. For math problems (LiveBench), generating multiple mutations for a single parent ($N$) was more effective. For retrieval tasks (HoVer), spreading mutations across more parents ($P$) yielded better results.
  • Diminishing Returns of Width: While parallelism is great, it's limited by the "slowest" call in a batch and the total size of the worker pool. The authors' model accurately predicts where these diminishing returns kick in.

Key Results

The batched approach outperformed the sequential baseline across the board:

  • Speed: Most batched runs finished in half the wall-clock time or less. The fastest configurations achieved a 3× to 4× speedup.
  • Performance:
    • On LiveBench-Math, scores rose from 68.9% to 72.1% (using a $2 \times 2$ configuration).
  • On HoVer, scores jumped significantly from 49.0% to 60.0% (using an $8 \times 1$ configuration).
  • Efficiency: Batched settings reached high validation scores much earlier in their "dollar budget" than sequential runs.
The main results show that parallel proposals are both faster and achieve higher test scores than the sequential baseline.

Limitations and Future Work

The authors identify that while they have successfully scaled the "width" of the optimization step ($P$ and $N$), there are other axes of scaling yet to be fully explored. These include the size of the mini-batch used for reflection and the amount of compute (e.g., chain-of-thought) spent on generating each mutation.

A major open question remains: given a fixed budget of dollars or API calls, how should a user automatically allocate that budget across these different "dials" (parents, mutations, mini-batch size, and reflection compute) to get the best possible agent? This work provides the first look at those trade-offs but leaves the automated allocation for future research.

Summary updated

Abstract

Reflective text optimization makes agentic systems better, but the optimization loop is itself a resource-hungry workload whose sequential structure leaves parallel evaluation capacity idle. We extend the open-source GEPA optimizer to propose and evaluate a batch of candidates on each optimization step instead of one candidate at a time. In our sweep on two tasks, most batched runs finished in half the wall-clock time or less, and the fastest in about a quarter to a third. Batched settings also achieved higher held-out test scores, from 68.9% to 72.1% on LiveBench-Math (with $2\times2$) and from 49.0% to 60.0% on HoVer (with $8\times1$). A simple runtime model explains the measured speedups under a finite worker pool, and batched settings also spend their metric-call and dollar budgets more efficiently. We frame the axes along which a reflective optimization step can scale in parallel, a first look at allocating a fixed optimization budget across them.
BibTeX
@misc{zhang2026batching,
  author = {Jialin Zhang and Lakshya A Agrawal and Shangyin Tan and Donghyun Lee and Lutfi Eren Erdogan and Dan Klein and Koushik Sen and Alex Dimakis and Matei Zaharia},
  title = {{Batching the Reflective Optimization Loop: Parallel Proposals Make GEPA Faster and Better}},
  year = {2026},
  eprint = {2610.36774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36774},
}
Figure