Abstract
Researchers from Baidu have developed a novel method to transform noisy, "low-quality" web-crawled mathematical data into high-quality training sets without relying on expensive human annotation or proprietary models like GPT-4. By training a "format converter" model on automatically aligned data pairs, they achieved a significant performance boost on Chinese math benchmarks, allowing a 7B parameter model to outperform much larger models and even GPT-3.5.
Introduction
To make Large Language Models (LLMs) excel at specific tasks like mathematical reasoning, high-quality Supervised Fine-Tuning (SFT) data is essential. Currently, researchers typically rely on expensive human labor or distillation from proprietary models like GPT-4. However, GPT-4 often struggles with specific domains and languages, such as Chinese mathematics.
While the web is a vast repository of math problems, this data is often "noisy." Optical Character Recognition (OCR) errors and formatting issues (e.g., $3^2$ becoming 32) create semantic inaccuracies that lead models to learn incorrect logic. This paper addresses the challenge of recycling this "trash" web data into "gold" training data through a model-based cleaning approach.
Methodology
The authors propose a four-step pipeline to create a "Model-based Data Cleaning" system, as illustrated in the figure below:
- Fuzzy Matching: A small "seed" dataset of high-quality math problems is aligned with a massive, messy web-crawled dataset. By matching questions or answers after stripping non-essential characters, the system creates
<low-quality, high-quality>pairs. - Training a Format Converter: A base LLM (such as Qwen or ChatGLM) is fine-tuned on these pairs to learn how to "translate" garbled web text into clean, LaTeX-formatted math.
- Mass Conversion: The trained converter is applied to the broader web-crawled dataset, transforming hundreds of thousands of noisy examples into a standardized format.
- Final Fine-Tuning: A fresh LLM is trained on the combined seed data and the newly "cleaned" web data.
The core intuition is that rewriting data is easier for an LLM than solving a problem from scratch; by using the noisy answer as a hint, the model can reconstruct the correct logic.
Experimental Setup
- Seed Dataset: 84,095 manually annotated instances.
- Web-Crawled Dataset: 573,960 instances collected from the web, containing significant formatting noise.
- Evaluation Benchmarks: Ape210K (Chinese elementary math) and CMATH (LLM-specific Chinese math benchmark).
Results and Analysis
The proposed method improved model performance by an average of 9.4% on Chinese math problems compared to using only high-quality seed data. Notably, the 7B parameter model (Ours-Qwen1.5) achieved an average score of 80.8, surpassing GPT-3.5-Turbo (73.6), Claude-2 (76.7), and DeepSeek-Chat 67B (78.5).
Key Insights
- Semantic Formatting: In mathematics, formatting is semantic. Model-based cleaning restores the lost dimensionality of formulas (e.g., superscripts) that rule-based scripts cannot infer.
- Base Model Advantage: Stronger base models (like Qwen) benefit more from this re-generation strategy, showing higher gains over rule-based cleaning compared to weaker models.
- Scaling Laws: Performance gains followed a log-linear relationship; doubling the amount of cleaned SFT data consistently improved scores.
Limitations and Future Work
While effective, the method currently requires a high-quality "seed" dataset for initial fuzzy matching. Future work may explore completely unsupervised cleaning or the addition of a mathematical verifier to ensure the logical correctness of the re-generated data. Additionally, the authors suggest expanding this "rewriting" paradigm to other complex domains like legal or medical records where web data is abundant but poorly structured.
Summary updated