TL;DR
When ensembling AI agents, more isn't always better—especially when those agents are just different "personas" of the same base model. This paper introduces BP-TD, a truth-discovery method that uses decorrelation weights to ensure a group of related models only gets "one vote," preventing a weak but large "bloc" from outvoting stronger, independent models.
What Problem Does This Paper Address?
In classical crowdsourcing, we assume workers make independent errors. If five people agree on an answer, it’s likely correct. However, modern AI "agents" are often highly correlated; they might be the same LLM (like Llama 3) with different system prompts or retrieval wrappers.
This creates a spurious majority. If you have six weak Llama-based agents and one strong Qwen-based agent, a standard majority vote will follow the Llama bloc even when they are all confidently wrong. Classical "truth discovery" algorithms, which reward agents for agreeing with the consensus, actually make this worse: they see the Llama bloc agreeing, label them as "reliable," and further discount the lone correct voice.
Methods
The authors propose BP-TD (Dependence-Aware Belief Propagation for Truth Discovery). It builds on the standard Dawid-Skene model—a probabilistic framework that iteratively estimates both the "truth" of an answer and the "reliability" of each source.
The key innovation is the addition of decorrelation weights ($w_a$). Instead of every agent getting a full vote, their influence is scaled down based on their membership in a "dependence group." The authors tested three ways to find these groups:
- Family (Structural): Grouping agents by their base model (e.g., all Llama 3.2 variants).
- Co-occurrence (Empirical): Grouping agents who agree with each other much more often than chance, regardless of whether they are right.
- Shared Error (Truth-aware): Grouping agents who make the same mistakes (this proved circular and less effective).
The process is visualized as a tripartite graph where belief flows between agents, questions, and candidate values.
Datasets
The researchers evaluated the method using a pool of 13 local agents (based on Llama 3.2, Qwen 2.5, Mistral, etc.) across four benchmarks:
- MMLU: A massive multitask language understanding benchmark (tested at scales of 160, 5,000, and 10,000 questions).
- ARC-Challenge: Grade-school science reasoning.
- TruthfulQA: Adversarial questions designed to mimic human misconceptions.
- SciQ: Grade-school science where most agents already perform near-perfectly.
Key Insights and Ideas
- The Reliability Inversion: Classical estimators completely fail on correlated AI agents. On MMLU, standard methods like Dawid-Skene actually invert the reliability ranking, labeling the weakest, most correlated agents as the most trustworthy because they agree with each other. BP-TD fixes this, achieving a Spearman correlation of +0.94 with true accuracy.
- The "Regime" Question: Decorrelation is a double-edged sword. It helps significantly when the largest group is weak (like the Llama bloc in their MMLU tests). However, if the largest group is actually the most accurate, decorrelation "throws away" good votes and can perform worse than a simple majority vote.
- Labels are King: One of the most striking findings is that just 20 ground-truth labels are more valuable than any unsupervised aggregation method. If you have a small budget for labels, using them to simply pick the single best agent is more accurate than complex ensemble algorithms.
Key Results
- MMLU Performance: On a 160-question sample, BP-TD achieved 0.716 accuracy, significantly beating the 0.624 of a simple majority vote.
- Consistency at Scale: The gains held up at scale; on 10,000 MMLU questions, BP-TD maintained a lead over majority vote (0.707 vs 0.674).
- Label-Free Success: BP-TD was the only aggregator to significantly beat majority vote on MMLU without using any labels for hyperparameter tuning.
Limitations and Future Work
The authors highlight a major "open problem": there is currently no label-free way to know if decorrelation will help or hurt a specific pool of agents. If you don't know whether your largest group is your strongest or your weakest, you are gambling by using decorrelation.
Additionally, the current model uses a heuristic "tempering" of the likelihood rather than a full generative model of agent dependence. Future work could involve more sophisticated Bayesian models to automatically infer these hidden dependency structures. Finally, the study focused on local models up to 14B parameters; the behavior of massive API-served models (like GPT-4 or Claude 3.5) remains to be characterized.
Summary updated