Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale

Jiale Liu, Yifan Zeng, Shaokun Zhang, Chi Zhang, Malte Højmark-Bertelsen, Marie Normann Gadeberg, Huazheng Wang, Qingyun WuarXiv 2025 · DOI · arXiv

Introduction

The rise of "LLM-as-Optimizer" has enabled researchers to use models like GPT-4 to iteratively refine an agent's instructions and toolsets by analyzing past failures. However, this approach faces a significant scalability bottleneck. As tasks grow in complexity, the resulting execution trajectories—logs of actions and observations—become too massive for standard context windows. Even when data fits, large language models (LLMs) often suffer from "lost in the middle" effects, failing to identify subtle patterns across a noisy, high-volume corpus.

To address these challenges, we present Fine-Grained Optimization (FGO), a framework that treats context limits as a structural constraint to be bypassed through a "divide and conquer" architecture.

The FGO Framework

FGO replaces traditional "all-at-once" optimization with a three-stage pipeline: Divide, Optimize, and Merge.

  1. Divide: The training dataset is partitioned into $N$ disjoint subsets. This allows the system to isolate specific task types (e.g., household cleaning vs. item retrieval), making the optimization problem more focused and manageable.
  2. Optimize: Each subset undergoes an iterative "Exploration-Evaluation-Optimization" cycle. An agent attempts the tasks, a post-hoc evaluator identifies failures, and an LLM optimizer generates a refined module (such as a specialized prompt) for that specific subset.
  3. Merge: To synthesize these local insights into a global solution, FGO employs Progressive Module Merging. Using recursive clustering, similar optimized modules are grouped and merged into intermediate versions. These are then back-tested against the training data to ensure quality, continuing until a single optimal module is produced.
The FGO pipeline: partitioning datasets into subsets, iteratively optimizing local modules, and progressively merging them into a final optimal system.

Experimental Evaluation

We evaluated FGO across three diverse benchmarks:

  • ALFWorld: Synthetic household tasks requiring multi-step reasoning.
  • GAIA: General assistant tasks involving web browsing and tool use.
  • LogisticsQA: A new dataset introduced in this work featuring real-world shipping invoices in complex XML formats, testing an agent's ability to handle extremely long-context documents.

Key Results and Efficiency

FGO demonstrates that fine-grained optimization is not only more accurate but significantly more efficient than traditional methods.

  • Superior Performance: FGO outperformed "all-at-once" optimization by 1.6% to 8.6% across all benchmarks. Compared to unoptimized agents, gains reached as high as 38.1%.
  • Token and Time Savings: By avoiding the quadratic cost of long-context prompts, FGO reduced average prompt token consumption by 56.3%. Furthermore, the parallelizable nature of the "Divide" step allowed FGO to reach 85% accuracy on ALFWorld 4x faster than sequential methods.
  • Scalability: Unlike batch-wise or bootstrapping methods, which often show erratic performance as data scales, FGO’s success rate improved steadily with larger training sets.
Performance vs. Token Cost across ALFWorld, LogisticsQA, and GAIA. FGO consistently achieves the highest accuracy with significantly lower prompt token usage compared to all-at-once methods.

Conclusion and Future Work

FGO provides a robust path forward for scaling agent optimization. While the current merging process introduces computational overhead due to back-testing, future work will explore using LLMs or Bayesian methods to predict merged module performance, further streamlining the pipeline. By preserving local expertise through subset optimization and distilling it via progressive merging, FGO ensures that agents remain both high-performing and cost-effective at scale.

Summary updated

Abstract

LLM-based optimization has shown remarkable potential in enhancing agentic systems. However, the conventional approach of prompting LLM optimizer with the whole training trajectories on training dataset in a single pass becomes untenable as datasets grow, leading to context window overflow and degraded pattern recognition. To address these challenges, we propose Fine-Grained Optimization (FGO), a scalable framework that divides large optimization tasks into manageable subsets, performs targeted optimizations, and systematically combines optimized components through progressive merging. Evaluation across ALFWorld, LogisticsQA, and GAIA benchmarks demonstrate that FGO outperforms existing approaches by 1.6-8.6% while reducing average prompt token consumption by 56.3%. Our framework provides a practical solution for scaling up LLM-based optimization of increasingly sophisticated agent systems. Further analysis demonstrates that FGO achieves the most consistent performance gain in all training dataset sizes, showcasing its scalability and efficiency.
BibTeX
@misc{liu2025divide,
  author = {Jiale Liu and Yifan Zeng and Shaokun Zhang and Chi Zhang and Malte Højmark-Bertelsen and Marie Normann Gadeberg and Huazheng Wang and Qingyun Wu},
  title = {{Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale}},
  year = {2025},
  eprint = {2505.03973},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.03973},
}
Figure