Diverse Multi-tool Aggregation with Large Language Models for Enhanced Math Reasoning

Bohan Yao, Vikas YadavEMNLP 2025 · OpenReview

TL;DR

This paper introduces Multi-TAG, a novel framework that significantly enhances Large Language Model (LLM) performance on complex math reasoning tasks by having the LLM concurrently invoke and aggregate outputs from multiple diverse tools at each reasoning step. This multi-tool aggregation approach leads to substantial accuracy improvements over existing single-tool methods, demonstrating the power of diverse tool utilization for robust mathematical problem-solving.

What Problem Does This Paper Address?

Large Language Models (LLMs) have shown impressive capabilities, but complex mathematical reasoning remains a significant challenge. While augmenting LLMs with external tools (like Python interpreters or WolframAlpha) has shown promise, existing approaches typically rely on the LLM selecting and using only one tool per reasoning step. This limitation becomes apparent when tackling more intricate math problems that require a combination of different reasoning strategies or cross-validation of intermediate results. These single-tool approaches struggle to achieve high accuracy on challenging benchmarks, and finetuning them for new LLMs is computationally expensive and sometimes impossible for proprietary models.

Methods

The core innovation is Multi-TAG (Multi-Tool AGgregation). Instead of a single tool invocation per step, Multi-TAG:

  1. Concurrent Multi-Tool Invocation: At each reasoning step, the LLM invokes multiple diverse tools (e.g., Chain-of-Thought reasoning, Python script execution, WolframAlpha queries).
  2. Candidate Generation: Each tool invocation generates a candidate for the next reasoning step, along with a final answer estimate.
  3. Aggregation and Selection: Multi-TAG aggregates these candidates. It identifies the most frequent final answer estimate across all tools. Candidates that agree on this majority answer are then shortlisted. From this shortlist, the candidate with the shortest solution completion (in terms of LLM tokens) is selected as the next step. This selection process aims for both accuracy and conciseness.
  4. Consistency Threshold: An early stopping mechanism is employed. If the "consistency gap" (difference between the most and second most frequent final answer estimates) exceeds a predefined threshold, the invocation for that step terminates early, saving computation.

Crucially, Multi-TAG is an inference-time, finetuning-free framework, making it highly adaptable to any LLM backbone, including proprietary ones.

Datasets

The paper evaluates Multi-TAG on four challenging short-answer math reasoning benchmarks:

  • MATH500: A subset of the full MATH dataset, focusing on higher difficulty levels.
  • AIME: American Invitational Mathematics Examination.
  • AMC: American Mathematics Competitions.
  • OlympiadBench: Specifically, the English text-only, open-ended splits for mathematics and physics.

Key Insights and Ideas

  • Diversity is Key: The paper strongly suggests that aggregating outputs from diverse tools is more effective than relying on a single tool, even if that single tool is highly capable. Different tools have different strengths and failure modes, and their agreement provides strong evidence of correctness.
  • Finetuning-Free Advantage: The inference-only nature of Multi-TAG is a major practical advantage. It bypasses the significant computational cost and limitations associated with finetuning LLMs, especially for proprietary models.
  • Conciseness Matters: Selecting the candidate with the shortest solution completion at each step not only leads to more efficient reasoning but also empirically improves performance, aligning with the "Occam's Razor" principle.
  • Cost-Performance Trade-off: The consistency threshold is a critical hyperparameter that allows users to balance computational cost with accuracy, significantly reducing token consumption without substantial performance loss.

Key Results

Multi-TAG consistently and substantially outperforms various baselines, including simple single-tool approaches and state-of-the-art tool-augmented LLM (TALM) frameworks.

  • Across three LLM backbones (LLaMA-3-70B, LLaMA-3.3-70B, and GPT-40) and four challenging benchmarks, Multi-TAG achieved average accuracy improvements of 6.0% to 7.5% over the strongest simple baselines and 7.9% to 13.7% over the strongest TALM baselines.
  • Even a simple aggregation of three tools (CoT + Python + WolframAlpha) without the full Multi-TAG selection mechanism showed significant gains, highlighting the benefit of multi-tool invocation.
  • Multi-TAG demonstrated consistent performance across different problem difficulties and subjects within the MATH500 dataset, performing particularly well on harder problems.

Limitations and Future Work

  • Generalization to Other Domains: The current work focuses on math reasoning. Future work could explore generalizing Multi-TAG to aggregate tools for other complex reasoning domains.
  • Tool Set Expansion: The study used a fixed set of three tools. Investigating the impact of adding more domain-specific tools is a promising avenue.
  • LLM Prompts: The paper acknowledges that the LLM prompts used for the TALM baselines were adapted from existing work, and further prompt engineering could potentially yield even better results for those baselines.

Summary updated

Abstract

Tool usage is a proven technique for developing high-performance reasoning in large language models (LLMs). Our work is focused on emphasizing the utility of leveraging multiple diverse tools for complex reasoning tasks. We present Multi-TAG, a Multi-Tool AGgregation-based LLM framework that utilizes multiple diverse tools to solve complex math problems over multiple reasoning steps. At each reasoning step, Multi-TAG invokes multiple tools and accepts the solution of the respective step by tools that have majority agreement on the final answer estimate. Multi-TAG strongly outperforms several standard baselines that use individual tools with the same number of runs, highlighting the importance of multi-tool invocation for solving complex reasoning tasks. We also show that naive aggregation of multiple tools at each reasoning step also leads to substantial improvements of up to 35% accuracy. Multi-TAG then further improves these gains by 7.4% on average on MATH500, AIME, AMC, and OlympiadBench.
BibTeX
@inproceedings{yao2025diverse,
  author = {Bohan Yao and Vikas Yadav},
  title = {{Diverse Multi-tool Aggregation with Large Language Models for Enhanced Math Reasoning}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=G96UB3JR8c},
}
Figure