Introduction
The rise of "LLM-as-Optimizer" has enabled researchers to use models like GPT-4 to iteratively refine an agent's instructions and toolsets by analyzing past failures. However, this approach faces a significant scalability bottleneck. As tasks grow in complexity, the resulting execution trajectories—logs of actions and observations—become too massive for standard context windows. Even when data fits, large language models (LLMs) often suffer from "lost in the middle" effects, failing to identify subtle patterns across a noisy, high-volume corpus.
To address these challenges, we present Fine-Grained Optimization (FGO), a framework that treats context limits as a structural constraint to be bypassed through a "divide and conquer" architecture.
The FGO Framework
FGO replaces traditional "all-at-once" optimization with a three-stage pipeline: Divide, Optimize, and Merge.
- Divide: The training dataset is partitioned into $N$ disjoint subsets. This allows the system to isolate specific task types (e.g., household cleaning vs. item retrieval), making the optimization problem more focused and manageable.
- Optimize: Each subset undergoes an iterative "Exploration-Evaluation-Optimization" cycle. An agent attempts the tasks, a post-hoc evaluator identifies failures, and an LLM optimizer generates a refined module (such as a specialized prompt) for that specific subset.
- Merge: To synthesize these local insights into a global solution, FGO employs Progressive Module Merging. Using recursive clustering, similar optimized modules are grouped and merged into intermediate versions. These are then back-tested against the training data to ensure quality, continuing until a single optimal module is produced.
Experimental Evaluation
We evaluated FGO across three diverse benchmarks:
- ALFWorld: Synthetic household tasks requiring multi-step reasoning.
- GAIA: General assistant tasks involving web browsing and tool use.
- LogisticsQA: A new dataset introduced in this work featuring real-world shipping invoices in complex XML formats, testing an agent's ability to handle extremely long-context documents.
Key Results and Efficiency
FGO demonstrates that fine-grained optimization is not only more accurate but significantly more efficient than traditional methods.
- Superior Performance: FGO outperformed "all-at-once" optimization by 1.6% to 8.6% across all benchmarks. Compared to unoptimized agents, gains reached as high as 38.1%.
- Token and Time Savings: By avoiding the quadratic cost of long-context prompts, FGO reduced average prompt token consumption by 56.3%. Furthermore, the parallelizable nature of the "Divide" step allowed FGO to reach 85% accuracy on ALFWorld 4x faster than sequential methods.
- Scalability: Unlike batch-wise or bootstrapping methods, which often show erratic performance as data scales, FGO’s success rate improved steadily with larger training sets.
Conclusion and Future Work
FGO provides a robust path forward for scaling agent optimization. While the current merging process introduces computational overhead due to back-testing, future work will explore using LLMs or Bayesian methods to predict merged module performance, further streamlining the pipeline. By preserving local expertise through subset optimization and distilling it via progressive merging, FGO ensures that agents remain both high-performing and cost-effective at scale.
Summary updated