AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation

Xianyang Liu, Yilin Liu, Shuai Wang, Hao Cheng, Andrew Estornell, Yuzhi Zhao, Jiaheng WeiSubmission to ICLR 2026 · Withdrawn

TL;DR

AgenticMath is a multi-agent framework designed to synthesize high-quality mathematical reasoning data for supervised fine-tuning (SFT) without relying on ground-truth solutions. By filtering seed questions, running an iterative rephrase–review–revise multi-agent loop, and ranking pairs based on quality and diversity, it produces AgenticMathQA (30K–90K samples). Models fine-tuned on this compact dataset match or outperform systems trained on massive datasets containing 400K to 2.3M samples across both in-domain and out-of-domain benchmarks.

What Problem Does This Paper Address?

Enhancing mathematical reasoning in open-weight large language models (LLMs) typically relies on supervised fine-tuning using massive synthetic datasets. However, existing data generation pipelines suffer from a fundamental bottleneck: quantity over quality.

Prior methods often expand datasets indiscriminately, synthesizing from noisy, ambiguous, or trivial seed questions and generating unverified solutions. In mathematics, problem formulation is decisive—poorly framed questions cannot yield sound multi-step reasoning traces. While multi-agent LLM systems have been explored for instruction tuning, they generally target shallow, open-ended conversational tasks rather than enforcing the strict logical consistency, solvability, and domain constraints required in mathematics.

Methods

AgenticMath enforces quality control across four distinct stages (powered by GPT-4o-mini):

[Raw Seeds (GSM8K/MATH)]
           │
           ▼
Stage 1: Seed Filtering ──► LLM scoring + k-NN consistency smoothing (τ ≥ 3)
           │
           ▼
Stage 2: Problem Rephrase ──► Multi-Agent Loop (Rephrase ➔ Review ➔ Revise; max 3 rounds)
           │
           ▼
Stage 3: Solution Generation ──► 1-shot Chain-of-Thought (CoT) generation (label-free)
           │
           ▼
Stage 4: Data Evaluation ──► Joint QA scoring + k-NN curation + long-tail diversity ranking
           │
           ▼
[Curated AgenticMathQA Dataset]
  1. Seed Problem Filtering: Candidate seed questions are scored across Complexity, Information Value, and Clarity & Precision (0–5). To eliminate evaluation noise, ratings are smoothed via a k-nearest neighbor ($k\text{-NN}$) consistency matrix in the embedding space before discarding low-quality seeds ($\tau < 3$).
  2. Agentic Problem Rephrase: Retained questions are expanded into 6 variations. A three-agent workflow refines them:
  3. Rephrase Agent: Introduces lexical richness and multi-step complexity.
  4. Review Agent: Evaluates grammar, logical coherence, and mathematical validity, outputting targeted critique.
  5. Revise Agent: Edits low-scoring candidates ($\tau_{\text{rev}} < 4.5$) based on critique in an iterative loop (up to 3 iterations).
  6. Solution Generation: A Solver Agent generates step-by-step reasoning traces for both seeds and rephrased problems via one-shot Chain-of-Thought (CoT) prompting, requiring no ground-truth reference solutions.
  7. Synthetic Data Evaluation & Long-Tail Ranking: Problem–solution pairs undergo joint quality scoring. Instead of a hard threshold, pairs are grouped into discrete score buckets and ranked within buckets using a long-tail diversity score to guarantee high semantic coverage.

Datasets

  • Seed Data: Human-authored problems from GSM8K (grade-school math) and MATH (competition-level high school math).
  • Introduced Dataset: AgenticMathQA, curated into 30K, 60K, and 90K subsets.
  • Evaluation Benchmarks (6 total):
  • In-Domain: GSM8K, MATH.
  • Out-of-Domain (OOD): CollegeMath (undergraduate math across 7 domains), DeepMind Mathematics (UK curriculum up to age 16), OlympiadBench-Math (challenging competition math), and TheoremQA (STEM theorem application).

Key Insights and Ideas

  • Curated Synthetic Questions Beat Human Seeds: In a direct 15K-sample comparison on Mistral-7B, training on purely synthetic AgenticMath questions outperformed the original human-authored seed questions (33.2 vs. 29.3 average score), with the largest jumps occurring on out-of-domain benchmarks.
  • The "Less Is More" Effect in Math SFT: Training on only 5%–15% of the data volume matches or beats models trained on 400K to 2M+ examples, demonstrating that dataset scale masks severe data redundancy and noise in existing corpora.
  • Algorithmic Design Trumps Teacher Model Strength: In controlled 10K experiments where competing methods (MetaMath, DART-Math, ScaleQuest) all used the exact same teacher (GPT-4o-mini), AgenticMath outperformed all baselines (31.6 average score vs. 26.3–29.9), confirming gains stem from quality filtering and iterative revision rather than teacher capability.
  • Diminishing Returns of Self-Reflection: Tracking quality metrics across review–revise iterations revealed that problem quality jumps markedly in rounds 1 and 2, but plateaus by round 3 (4.06 $\rightarrow$ 4.13 $\rightarrow$ 4.13), justifying early stopping.

Key Results

AgenticMath was evaluated across multiple 3B–8B base architectures (Qwen2.5-3B, DeepSeekMath-7B, Mistral-7B, Llama3-8B):

  • Small-Scale SOTA (30K–60K):
  • Qwen2.5-3B: 30K AgenticMath achieved 53.7 average accuracy, outperforming 30K MathFusion (38.1) by over 15 points. At 60K, it reached 53.9.
  • Mistral-7B: 60K AgenticMath achieved 38.4, outperforming 60K MathFusion (36.1) and 60K DART-Math (32.9).
  • Outperforming Massive Baselines (400K–2.3M):
  • Llama3-8B: AgenticMath with 90K samples reached 40.3 average accuracy, surpassing Llama3-8B-DART-Math trained on 590K samples (39.7) and Llama3-8B-MMIQC trained on 2.3M samples (35.6).
  • DeepSeekMath-7B: AgenticMath with 60K samples reached 49.3 average accuracy, beating DeepSeekMath-7B-RFT trained on 590K samples (48.3) and DeepSeekMath-7B-Instruct trained on 780K samples (43.5).

Limitations and Future Work

  • Exploration Scale: Experiments were restricted to models between 3B and 8B parameters; scaling behavior on larger open models (e.g., 70B+) remains unverified.
  • Teacher Dependency: Generating and reviewing questions still relies on commercial proprietary LLMs (GPT-4o-mini), meaning data generation is bound to the capabilities and API limits of the teacher.
  • Inference Compute vs. Pre-filtering Cost: While downstream fine-tuning is data- and compute-efficient, running multi-agent review–revise loops (expanding seeds $6\times$ with up to 3 evaluation rounds) introduces upfront inference overhead during dataset construction.

Summary updated

Abstract

The creation of high-quality datasets to improve Large Language Model (LLM) reasoning remains a significant challenge, as current methods often suffer from generating low-quality/incorrect answers and limited information richness from available data sources. To address this, we propose AgenticMath, a novel agentic pipeline for generating high-quality mathematical question-answer pairs to enhance the supervised fine-tuning of LLMs. Our method operates through four stages: (1) Seed Question Filter that selects questions with high information richness, complexity, and clarity; (2) an Agentic Question Rephrase step that employs a multi-agent system to generate diverse, logically consistent paraphrases; (3) an Answer Augment step where rewrite answers using chain-of-thought reasoning to enhance numerical and logical correctness, without reliance on human-provided labels; and (4) a final Question and Answer Evaluation that retains only the most superior pairs. Extensive experiments demonstrate that, fine-tuning 3B-8B parameter LLMs on AgenticMath generated datasets (comprising only 30-60K math samples) achieves competitive or superior performance on diverse in domain and out-of-domain mathematical reasoning benchmarks compared to baselines trained on much more data (e.g., 400K or 2.3M samples). Our work demonstrates that targeted, high-quality data generation is a more efficient path to improving mathematical reasoning in LLMs than large-scale, low-quality alternatives.
BibTeX
@misc{liu2025agenticmath,
  author = {Xianyang Liu and Yilin LIU and Shuai Wang and Hao Cheng and Andrew Estornell and Yuzhi Zhao and Jiaheng Wei},
  title = {{AgenticMath: Enhancing LLM Reasoning via Agentic-based Math Data Generation}},
  year = {2025},
  eprint = {2510.19361},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.19361},
}
Figure