RA-MoWE: Workflow-Affinity Embeddings for Query Clustering and Agentic Workflow Generation

Qi Cheng, Shengyu Chen, Wei Cheng, Yiqun Xie, Haoyu Wang, Haifeng Chen, Xiaowei JiaarXiv 2026 · arXiv

TL;DR

RA-MoWE introduces workflow-affinity embeddings to group LLM queries by their computational needs rather than just their semantic similarity. By clustering queries based on how they respond to different reasoning strategies (like Chain-of-Thought or ReAct), the framework generates specialized, reusable "expert" workflows that outperform general-purpose agents while reducing inference costs by nearly 28%.

What Problem Does This Paper Address?

Agentic workflows—which coordinate LLM reasoning, tool use, and verification—are powerful but notoriously difficult to design. Currently, researchers face a frustrating tradeoff:

  1. One-size-fits-all: Optimize a single workflow for an entire dataset. This is efficient but ignores the fact that a math word problem requires different reasoning than a Python debugging task.
  2. Per-query search: Search for a custom workflow for every single query. This is highly effective but prohibitively expensive in terms of time and API costs.

Furthermore, standard text embeddings (like those used in RAG) capture what a query is about, but they don't necessarily capture how it should be solved. Two queries about "sorting algorithms" might look similar, but one might need a simple code generation step while the other requires complex multi-step verification.

Methods

The authors propose a three-stage framework called RA-MoWE (Retrieval-Augmented Mixture of Workflow Experts):

  1. Offline Probing & Embedding (C1): The system runs a panel of 12 "probe" workflows (e.g., Direct, CoT, Self-Refine, RAG, Tool Use) on training queries. Each query is assigned a workflow-affinity embedding—a vector where each coordinate represents how well a specific probe solved that query.
  2. Clustering & Expert Generation (C3): Queries are clustered based on these embeddings. For each cluster, the system uses the cluster's "average" profile to initialize a specialized workflow. It then uses an LLM-based optimizer to refine this workflow through execution feedback, creating a bank of reusable experts.
  3. Predictive Routing (C2): To avoid running expensive probes at inference time, the authors train an embedding encoder (a small MLP on top of frozen BGE features). This encoder learns to predict a query's affinity embedding directly from its raw text.

At deployment, a new query's text is fed to the encoder, which selects the nearest cluster's expert workflow to execute.

Figure 2: The RA-MoWE framework. Offline, reference-workflow scores define query embeddings for clustering, expert generation, and encoder supervision. At deployment, the encoder predicts an embedding from query text and selects a stored expert. Dashed arrows indicate saved supervision and experts. C1–C3 mark the embedding, encoder, and generation contributions.

Datasets

The framework was evaluated on two primary targets:

  • MixBench-H: A diverse 900-query collection (600 train / 300 test) spanning AIME (math), GPQA (science), CodeContests, and LiveCodeBench (programming).
  • SWE-bench Verified: A benchmark of 500 real-world software engineering issues from GitHub, requiring repository-level localization and repair.

Key Insights and Ideas

  • Behavioral Similarity > Semantic Similarity: The core "aha moment" is that queries should be grouped by their response to reasoning strategies. As shown in Figure 1, different clusters exhibit distinct "affinity profiles"—some thrive on self-refinement, while others strictly require tool use.
  • Amortized Optimization: By searching for experts at the cluster level rather than the query level, RA-MoWE captures the benefits of specialized search while spreading the "cost" of that search across all future queries in that cluster.
  • The Encoder as a Shortcut: The ability to predict these complex behavioral embeddings from raw text is what makes the system practical for real-time serving.
Figure 1: Different workflow preferences motivate specialized experts. (a) Average embeddings for six GPT-4o-mini training-query clusters show distinct responses to reference workflows. C2 fa- vors sampling and answer selection (CoT-SC), whereas C4 favors self-refinement. (b) RA-MoWE uses these preferences to choose initial workflows. Values are standardized scores centered within each row; outlines mark row maxima, and parentheses give full cluster sizes. Refine revises a chain-of-thought (CoT) answer; Ens.(n) selects among n sampled CoT answers; Plan adds decom- position. Asterisks mark runnable substitutes for unavailable workflows. Measurement details are in Appendix C.4.

Key Results

  • Performance Boost: On MixBench-H, RA-MoWE improved the average task score by 4.04 percentage points over simply selecting the best reference workflow.
  • Efficiency Gains: The system used 27.7% fewer LLM calls at inference compared to the routed probe baseline (2.11 vs. 2.92 calls per query).
  • State-of-the-Art in Specialized Tasks: On SWE-bench Verified, RA-MoWE achieved a 24.5% resolution rate, the highest among tested adaptations, outperforming established methods like AFlow and ScoreFlow.
Figure 3: Embedding-guided expert generation on 300 MixBench-H queries with three executions per arm. Left: utility versus serving calls/query. Right: Guided-B utility differences with ex- ploratory 95% paired query-bootstrap intervals, resampling per-query execution means. Guided-B improves on Direct and routed probes; its difference from single-call CoT is unresolved. Serving calls exclude offline measurement and construction. Prompting, decoding, and operator differences make the contrasts comparisons of complete configurations.

Limitations and Future Work

  • Encoder Fidelity: The authors note that the current text-to-embedding predictors are not perfect. If the encoder assigns a query to the wrong cluster, the resulting expert might be less effective than a general-purpose one.
  • Probe Panel Dependency: The quality of the experts is inherently limited by the diversity of the initial probe panel. If no probe captures a specific necessary reasoning style, the generated experts may remain sub-optimal.
  • Static Expert Bank: Currently, experts are generated offline and remain fixed. Future iterations could explore "online" experts that continue to learn and refine themselves as they encounter more deployment data.

Summary updated

Abstract

Agentic workflows enable large language models (LLMs) to solve complex tasks by coordinating reasoning, tool use, and verification. However, a workflow optimized for an entire task collection can overlook differences in the reasoning strategies that individual queries need, while searching for a new workflow for every query repeats costly optimization. To address this tradeoff, we introduce RA-MoWE, a framework that uses workflow-affinity embeddings to cluster queries and guide the generation of reusable expert workflows. Each embedding records how well a fixed set of reference workflows solves a query, revealing similarities in which reasoning strategies are effective. RA-MoWE uses each cluster's queries and average embedding to initialize and refine a specialized workflow through execution feedback. An embedding encoder predicts these embeddings from query text, allowing new queries to select a generated expert without first executing the reference workflows. On a 300-query test set drawn from four benchmarks spanning mathematics, science, and programming, RA-MoWE improves average task score by 4.04 percentage points over selecting among the reference workflows, while using 27.7% fewer language-model calls at inference.
BibTeX
@misc{cheng2026ra,
  author = {Qi Cheng and Shengyu Chen and Wei Cheng and Yiqun Xie and Haoyu Wang and Haifeng Chen and Xiaowei Jia},
  title = {{RA-MoWE: Workflow-Affinity Embeddings for Query Clustering and Agentic Workflow Generation}},
  year = {2026},
  eprint = {2610.07851},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.07851},
}

Loading related content...

Figure