TL;DR
AgenticMath is a multi-agent framework designed to synthesize high-quality mathematical reasoning data for supervised fine-tuning (SFT) without relying on ground-truth solutions. By filtering seed questions, running an iterative rephrase–review–revise multi-agent loop, and ranking pairs based on quality and diversity, it produces AgenticMathQA (30K–90K samples). Models fine-tuned on this compact dataset match or outperform systems trained on massive datasets containing 400K to 2.3M samples across both in-domain and out-of-domain benchmarks.
What Problem Does This Paper Address?
Enhancing mathematical reasoning in open-weight large language models (LLMs) typically relies on supervised fine-tuning using massive synthetic datasets. However, existing data generation pipelines suffer from a fundamental bottleneck: quantity over quality.
Prior methods often expand datasets indiscriminately, synthesizing from noisy, ambiguous, or trivial seed questions and generating unverified solutions. In mathematics, problem formulation is decisive—poorly framed questions cannot yield sound multi-step reasoning traces. While multi-agent LLM systems have been explored for instruction tuning, they generally target shallow, open-ended conversational tasks rather than enforcing the strict logical consistency, solvability, and domain constraints required in mathematics.
Methods
AgenticMath enforces quality control across four distinct stages (powered by GPT-4o-mini):
[Raw Seeds (GSM8K/MATH)]
│
▼
Stage 1: Seed Filtering ──► LLM scoring + k-NN consistency smoothing (τ ≥ 3)
│
▼
Stage 2: Problem Rephrase ──► Multi-Agent Loop (Rephrase ➔ Review ➔ Revise; max 3 rounds)
│
▼
Stage 3: Solution Generation ──► 1-shot Chain-of-Thought (CoT) generation (label-free)
│
▼
Stage 4: Data Evaluation ──► Joint QA scoring + k-NN curation + long-tail diversity ranking
│
▼
[Curated AgenticMathQA Dataset]
- Seed Problem Filtering: Candidate seed questions are scored across Complexity, Information Value, and Clarity & Precision (0–5). To eliminate evaluation noise, ratings are smoothed via a k-nearest neighbor ($k\text{-NN}$) consistency matrix in the embedding space before discarding low-quality seeds ($\tau < 3$).
- Agentic Problem Rephrase: Retained questions are expanded into 6 variations. A three-agent workflow refines them:
- Rephrase Agent: Introduces lexical richness and multi-step complexity.
- Review Agent: Evaluates grammar, logical coherence, and mathematical validity, outputting targeted critique.
- Revise Agent: Edits low-scoring candidates ($\tau_{\text{rev}} < 4.5$) based on critique in an iterative loop (up to 3 iterations).
- Solution Generation: A Solver Agent generates step-by-step reasoning traces for both seeds and rephrased problems via one-shot Chain-of-Thought (CoT) prompting, requiring no ground-truth reference solutions.
- Synthetic Data Evaluation & Long-Tail Ranking: Problem–solution pairs undergo joint quality scoring. Instead of a hard threshold, pairs are grouped into discrete score buckets and ranked within buckets using a long-tail diversity score to guarantee high semantic coverage.
Datasets
- Seed Data: Human-authored problems from GSM8K (grade-school math) and MATH (competition-level high school math).
- Introduced Dataset: AgenticMathQA, curated into 30K, 60K, and 90K subsets.
- Evaluation Benchmarks (6 total):
- In-Domain: GSM8K, MATH.
- Out-of-Domain (OOD): CollegeMath (undergraduate math across 7 domains), DeepMind Mathematics (UK curriculum up to age 16), OlympiadBench-Math (challenging competition math), and TheoremQA (STEM theorem application).
Key Insights and Ideas
- Curated Synthetic Questions Beat Human Seeds: In a direct 15K-sample comparison on Mistral-7B, training on purely synthetic AgenticMath questions outperformed the original human-authored seed questions (33.2 vs. 29.3 average score), with the largest jumps occurring on out-of-domain benchmarks.
- The "Less Is More" Effect in Math SFT: Training on only 5%–15% of the data volume matches or beats models trained on 400K to 2M+ examples, demonstrating that dataset scale masks severe data redundancy and noise in existing corpora.
- Algorithmic Design Trumps Teacher Model Strength: In controlled 10K experiments where competing methods (MetaMath, DART-Math, ScaleQuest) all used the exact same teacher (GPT-4o-mini), AgenticMath outperformed all baselines (31.6 average score vs. 26.3–29.9), confirming gains stem from quality filtering and iterative revision rather than teacher capability.
- Diminishing Returns of Self-Reflection: Tracking quality metrics across review–revise iterations revealed that problem quality jumps markedly in rounds 1 and 2, but plateaus by round 3 (4.06 $\rightarrow$ 4.13 $\rightarrow$ 4.13), justifying early stopping.
Key Results
AgenticMath was evaluated across multiple 3B–8B base architectures (Qwen2.5-3B, DeepSeekMath-7B, Mistral-7B, Llama3-8B):
- Small-Scale SOTA (30K–60K):
- Qwen2.5-3B: 30K AgenticMath achieved 53.7 average accuracy, outperforming 30K MathFusion (38.1) by over 15 points. At 60K, it reached 53.9.
- Mistral-7B: 60K AgenticMath achieved 38.4, outperforming 60K MathFusion (36.1) and 60K DART-Math (32.9).
- Outperforming Massive Baselines (400K–2.3M):
- Llama3-8B: AgenticMath with 90K samples reached 40.3 average accuracy, surpassing Llama3-8B-DART-Math trained on 590K samples (39.7) and Llama3-8B-MMIQC trained on 2.3M samples (35.6).
- DeepSeekMath-7B: AgenticMath with 60K samples reached 49.3 average accuracy, beating DeepSeekMath-7B-RFT trained on 590K samples (48.3) and DeepSeekMath-7B-Instruct trained on 780K samples (43.5).
Limitations and Future Work
- Exploration Scale: Experiments were restricted to models between 3B and 8B parameters; scaling behavior on larger open models (e.g., 70B+) remains unverified.
- Teacher Dependency: Generating and reviewing questions still relies on commercial proprietary LLMs (GPT-4o-mini), meaning data generation is bound to the capabilities and API limits of the teacher.
- Inference Compute vs. Pre-filtering Cost: While downstream fine-tuning is data- and compute-efficient, running multi-agent review–revise loops (expanding seeds $6\times$ with up to 3 evaluation rounds) introduces upfront inference overhead during dataset construction.
Summary updated