TL;DR
RA-MoWE introduces workflow-affinity embeddings to group LLM queries by their computational needs rather than just their semantic similarity. By clustering queries based on how they respond to different reasoning strategies (like Chain-of-Thought or ReAct), the framework generates specialized, reusable "expert" workflows that outperform general-purpose agents while reducing inference costs by nearly 28%.
What Problem Does This Paper Address?
Agentic workflows—which coordinate LLM reasoning, tool use, and verification—are powerful but notoriously difficult to design. Currently, researchers face a frustrating tradeoff:
- One-size-fits-all: Optimize a single workflow for an entire dataset. This is efficient but ignores the fact that a math word problem requires different reasoning than a Python debugging task.
- Per-query search: Search for a custom workflow for every single query. This is highly effective but prohibitively expensive in terms of time and API costs.
Furthermore, standard text embeddings (like those used in RAG) capture what a query is about, but they don't necessarily capture how it should be solved. Two queries about "sorting algorithms" might look similar, but one might need a simple code generation step while the other requires complex multi-step verification.
Methods
The authors propose a three-stage framework called RA-MoWE (Retrieval-Augmented Mixture of Workflow Experts):
- Offline Probing & Embedding (C1): The system runs a panel of 12 "probe" workflows (e.g., Direct, CoT, Self-Refine, RAG, Tool Use) on training queries. Each query is assigned a workflow-affinity embedding—a vector where each coordinate represents how well a specific probe solved that query.
- Clustering & Expert Generation (C3): Queries are clustered based on these embeddings. For each cluster, the system uses the cluster's "average" profile to initialize a specialized workflow. It then uses an LLM-based optimizer to refine this workflow through execution feedback, creating a bank of reusable experts.
- Predictive Routing (C2): To avoid running expensive probes at inference time, the authors train an embedding encoder (a small MLP on top of frozen BGE features). This encoder learns to predict a query's affinity embedding directly from its raw text.
At deployment, a new query's text is fed to the encoder, which selects the nearest cluster's expert workflow to execute.
Datasets
The framework was evaluated on two primary targets:
- MixBench-H: A diverse 900-query collection (600 train / 300 test) spanning AIME (math), GPQA (science), CodeContests, and LiveCodeBench (programming).
- SWE-bench Verified: A benchmark of 500 real-world software engineering issues from GitHub, requiring repository-level localization and repair.
Key Insights and Ideas
- Behavioral Similarity > Semantic Similarity: The core "aha moment" is that queries should be grouped by their response to reasoning strategies. As shown in Figure 1, different clusters exhibit distinct "affinity profiles"—some thrive on self-refinement, while others strictly require tool use.
- Amortized Optimization: By searching for experts at the cluster level rather than the query level, RA-MoWE captures the benefits of specialized search while spreading the "cost" of that search across all future queries in that cluster.
- The Encoder as a Shortcut: The ability to predict these complex behavioral embeddings from raw text is what makes the system practical for real-time serving.
Key Results
- Performance Boost: On MixBench-H, RA-MoWE improved the average task score by 4.04 percentage points over simply selecting the best reference workflow.
- Efficiency Gains: The system used 27.7% fewer LLM calls at inference compared to the routed probe baseline (2.11 vs. 2.92 calls per query).
- State-of-the-Art in Specialized Tasks: On SWE-bench Verified, RA-MoWE achieved a 24.5% resolution rate, the highest among tested adaptations, outperforming established methods like AFlow and ScoreFlow.
Limitations and Future Work
- Encoder Fidelity: The authors note that the current text-to-embedding predictors are not perfect. If the encoder assigns a query to the wrong cluster, the resulting expert might be less effective than a general-purpose one.
- Probe Panel Dependency: The quality of the experts is inherently limited by the diversity of the initial probe panel. If no probe captures a specific necessary reasoning style, the generated experts may remain sub-optimal.
- Static Expert Bank: Currently, experts are generated offline and remain fixed. Future iterations could explore "online" experts that continue to learn and refine themselves as they encounter more deployment data.
Summary updated