The Avengers: A Routing Recipe for Collective Intelligence in Language Models

Yiqun Zhang, Hui Li, Chenxu Wang, Ling Chen, Qiang Zhang, Peng Ye, Shi Feng, Xinrun Wang, Jia Xu, Lei Bai, Shuyue Hu

TL;DR

This paper introduces "The Avengers," a novel framework that enables a collection of smaller, open-source language models to collectively outperform leading proprietary models like GPT-4. By intelligently routing queries to specialized models based on semantic clustering and performance profiling, Avengers achieves state-of-the-art results across diverse tasks without requiring any neural network training.

What Problem Does This Paper Address?

The current landscape of large language models (LLMs) is increasingly dominated by massive, proprietary models that require immense computational resources. This creates a barrier for open-source communities and researchers who often work with smaller, more accessible models. While individual small models can excel in specific niches, the question remains: can these smaller models, when orchestrated effectively, compete with the general capabilities of their larger, proprietary counterparts across a wide array of tasks? The authors aim to bridge this gap by demonstrating a method to harness the collective intelligence of multiple small LLMs, democratizing access to high-performance AI.

Methods

The Avengers framework operates in two phases: offline calibration and online inference, all without requiring any neural network training.

Offline Calibration: 1. Embedding: Queries from a validation set are encoded into semantic vector representations using a text embedding model. 2. Clustering: These semantic vectors are clustered (e.g., using K-Means) into $k$ groups, where each cluster represents a distinct type of query. 3. Scoring: Each available small language model (SLM) is evaluated on the validation set for its performance within each cluster. This generates a "capability profile" for each SLM, indicating its strengths and weaknesses across different query types. 4. Model Set Construction: The framework automatically selects a complementary set of SLMs from a larger pool based on their cluster-wise performance profiles, aiming for diversity and specialization.

Online Inference: 1. Embedding & Routing: An incoming test query is embedded using the same model as in calibration. It's then routed to the nearest cluster based on its semantic representation. 2. Model Selection: The SLM(s) with the highest performance scores within that specific cluster (as determined by the capability profiles) are selected. 3. Voting: The selected model(s) generate responses, and a final answer is produced through repeated sampling and majority voting (e.g., Self-Consistency or Model-Switch).

Datasets

The paper evaluates Avengers on 15 diverse datasets spanning five categories:

  • Mathematics: AIME, Math500, LiveMathBench
  • Code: MBPP, HumanEval
  • Logic: KORBench, Knights and Knaves, BBH
  • Knowledge: ARC Challenge, MMLUPro, GPQA, FinQA, MedQA
  • Affective: EmoryNLP, MELD

For evaluating out-of-distribution (OOD) generalization, they introduce five additional datasets, one for each category: MathBench, StudentEval, Winogrande, BRAIN-TEASER, and DailyDialog. These OOD datasets are not used for clustering or validation.

Key Insights and Ideas

  • Collective Intelligence Trumps Size: The core idea is that a well-orchestrated ensemble of smaller models can achieve superior performance to a single, larger model, even proprietary ones. This challenges the "bigger is always better" paradigm.
  • No Training Required: Avengers achieves its performance without any neural network training, making it highly adaptable and reproducible. This significantly lowers the barrier to entry for using and improving such systems.
  • "Horses for Courses" Routing: The framework's effectiveness hinges on identifying the right model for the right query type. By clustering queries and profiling model capabilities, Avengers ensures that specialized models are leveraged for their strengths.
  • Robustness to Hyperparameters and Components: The framework demonstrates remarkable robustness across various embedding models, clustering algorithms, and ensemble strategies, highlighting its generalizability.
  • Automatic Model Selection: The ability to automatically select a diverse and complementary set of SLMs from a larger pool is crucial for achieving broad performance without human expertise.

Key Results

  • Surpasses GPT-4 Variants: With 10 open-source models (~7B parameters each), Avengers achieves an average score of 70.54 across 15 datasets, outperforming GPT-4o (60.22), GPT-4.1 (69.20), and GPT-4.5 (68.18).
  • Significant Gains in Specific Tasks: Avengers shows notable improvements, such as +37.04% on AIME and +31.86% on MBPP compared to GPT-4.1.
  • Superior OOD Generalization: Avengers outperforms router-based baselines on out-of-distribution tasks by an average of 8.56%.
  • Efficiency with Few Models: Avengers can match GPT-4.1/4.5 performance using just three automatically selected models from a pool of 22.
  • Data Efficiency: The framework achieves strong performance even with a small portion (30%) of data for clustering.

Limitations and Future Work

  • Mixture-Based Methods Struggle with Small Models: The paper notes that mixture-based approaches, while effective for larger models, perform poorly with smaller models due to their limitations in instruction following and context handling.
  • GPT-4.1 Advantage in Knowledge-Intensive Tasks: Proprietary models like GPT-4.1 still maintain an advantage in highly specialized knowledge-intensive tasks, likely due to their larger parameter size and broader knowledge base.
  • Future Work: The authors suggest that as more models become available, the complementary strengths of SLMs will continue to emerge, further enhancing performance, especially in knowledge, code, and affective tasks. The framework's adaptability also positions it well for incorporating new models and datasets as the field evolves.

Summary updated

Abstract

Proprietary models are increasingly dominating the race for ever-larger language models. Can open-source, smaller models remain competitive across a broad range of tasks? In this paper, we present the Avengers---a lightweight framework that leverages the collective intelligence of these smaller models. The Avengers builds upon four lightweight operations: (i) embedding: encode queries using a text embedding model; (ii) clustering: group queries based on their semantic similarity; (iii) scoring: scores each model's performance within each cluster; and (iv) voting: improve outputs via repeated sampling and voting. At inference time, each query is embedded and assigned to its nearest cluster. The top-performing model(s) within that cluster are selected to generate the response with repeated sampling. Remarkably, with 10 open-source models (~7B parameters each), the Avengers surpasses GPT-4o, 4.1, and 4.5 in average performance across 15 diverse datasets spanning mathematics, coding, logical reasoning, general knowledge, and affective tasks. In particular, it surpasses GPT-4.1 on mathematics tasks by 18.21% and on code tasks by 7.46%. Furthermore, the Avengers delivers superior out-of-distribution generalization, and remains robust across various embedding models, clustering algorithms, ensemble strategies, data efficiency, and values of its sole parameter---the number of clusters.
BibTeX
@inproceedings{zhang2026avengers,
  author = {Yiqun Zhang and Hui Li and Chenxu Wang and Ling Chen and Qiang Zhang and Peng Ye and Shi Feng and Xinrun Wang and Jia Xu and Lei Bai and Shuyue Hu},
  title = {{The Avengers: A Routing Recipe for Collective Intelligence in Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  doi = {10.1609/aaai.v40i41.40790},
}
Figure