Autoformalizer with Tool Feedback

Qi Guo, Jianing Wang, Jianfei Zhang, Deyang Kong, Xiangzhou Huang, Xiangyu Xi, Wei Wang, Jingang Wang, Xunliang Cai, Shikun Zhang, Wei Ye

Abstract

Autoformalization addresses the scarcity of data for Automated Theorem Proving (ATP) by translating mathematical problems from natural language into formal statements. Efforts in recent work shift from directly prompting large language models to training an end-to-end formalizer model from scratch, achieving remarkable advancements. However, existing formalizer still struggles to consistently generate valid statements that meet syntactic validity and semantic consistency. To address this issue, we propose the Autoformalizer with Tool Feedback (ATF), a novel approach that incorporates syntactic and consistency information as tools into the formalization process. By integrating Lean 4 compilers for syntax corrections and employing a multi-LLMs-as-judge approach for consistency validation, the model is able to adaptively refine generated statements according to the tool feedback, enhancing both syntactic validity and semantic consistency. The training of ATF involves a cold-start phase on synthetic tool-calling data, an expert iteration phase to improve formalization capabilities, and Direct Preference Optimization to alleviate ineffective revisions. Experimental results show that ATF markedly outperforms a range of baseline formalizer models, with its superior performance further validated by human evaluations. Subsequent analysis reveals that ATF demonstrates excellent inference scaling properties. Moreover, we open-source Numina-ATF, a dataset containing 750K synthetic formal statements to facilitate advancements in autoformalization and ATP research.

Introduction

The development of Automated Theorem Proving (ATP) systems is currently bottlenecked by the extreme scarcity of formal mathematical data. While the internet provides an abundance of "informal" mathematics in the form of textbooks and LaTeX documents, "formal" mathematics—code that can be verified by a computer—is difficult to produce. Autoformalization aims to bridge this gap by translating natural language math problems into formal languages like Lean 4.

However, autoformalization faces two primary hurdles: syntactic validity and semantic consistency. Formal languages are notoriously unforgiving; a single syntax error renders a statement useless. More subtly, a model may produce syntactically correct code that misinterprets the original problem, such as confusing existential and universal quantification or using incorrect constants.

Common pitfalls in autoformalization, including logical errors and parameter mismatches.

Methodology: Autoformalizer with Tool Feedback (ATF)

To overcome these challenges, we introduce the Autoformalizer with Tool Feedback (ATF). Rather than treating formalization as a one-shot translation task, ATF treats it as an iterative debugging process where the model interacts with external tools to refine its output.

Tool Integration

ATF utilizes two primary tools:

  • Syntax Check: The model interacts with a Lean 4 compiler. To optimize this process, we implemented a "grouped execution" method that batches multiple statements into a single compiler pass, reducing computational overhead by 87%.
  • Consistency Check: To ensure semantic accuracy, we employ a multi-LLM-as-judge approach. Multiple models compare the informal text against the formal code to detect misalignments, significantly reducing false positives compared to single-model evaluation.

Training Pipeline

The training of ATF follows a structured three-stage process:

  • Cold-Start: The model is trained on synthetic trajectories to learn the basic loop of thought, tool calling, and refinement.
  • Expert Iteration: The model practices on a dataset of 178,000 competition-level problems, retaining successful "debugging" paths for fine-tuning.
  • Direct Preference Optimization (DPO): This stage rewards trajectories that reach correct solutions efficiently, preventing the model from falling into ineffective revision loops.
The ATF training and inference pipeline, showcasing the iterative feedback loop between the formalizer and verification tools.

Experimental Results

ATF-32B demonstrates state-of-the-art performance across multiple benchmarks. On CombiBench, a challenging combinatorial math dataset, ATF-32B achieved a semantic consistency score of 65.38%, representing a 29.13% improvement over previous leaders.

Furthermore, the model exhibits strong inference-time scaling properties. By allowing more revision attempts during inference, the model's accuracy continues to improve, effectively trading additional compute for higher precision. Our distilled 8B model also showed remarkable efficiency, outperforming several larger 32B baseline models. Human validation confirmed that the consistency check tool has a high correlation (0.746) with expert judgment.

Conclusion

ATF represents a significant step forward in making autoformalization more robust and reliable. By teaching models to "compile" and verify their own mathematical translations, we can generate high-quality formal data at scale. Future work will focus on extending this iterative feedback loop from problem statements to the formalization of complete proofs. To support the community, we have open-sourced Numina-ATF, a dataset of 750,000 synthetic formal statements.

Summary updated

Abstract

Autoformalization addresses the scarcity of data for Automated Theorem Proving (ATP) by translating mathematical problems from natural language into formal statements. Efforts in recent work shift from directly prompting large language models to training an end-to-end formalizer model from scratch, achieving remarkable advancements. However, existing formalizer still struggles to consistently generate valid statements that meet syntactic validity and semantic consistency. To address this issue, we propose the Autoformalizer with Tool Feedback (ATF), a novel approach that incorporates syntactic and consistency information as tools into the formalization process. By integrating Lean 4 compilers for syntax corrections and employing a multi-LLMs-as-judge approach for consistency validation, the model is able to adaptively refine generated statements according to the tool feedback, enhancing both syntactic validity and semantic consistency. The training of ATF involves a cold-start phase on synthetic tool-calling data, an expert iteration phase to improve formalization capabilities, and Direct Preference Optimization to alleviate ineffective revisions. Experimental results show that ATF markedly outperforms a range of baseline formalizer models, with its superior performance further validated by human evaluations. Subsequent analysis reveals that ATF demonstrates excellent inference scaling properties. Moreover, we open-source Numina-ATF, a dataset containing 750K synthetic formal statements to facilitate advancements in autoformalization and ATP research.
BibTeX
@misc{guo2025autoformalizer,
  author = {Qi Guo and Jianing Wang and Jianfei Zhang and Deyang Kong and Xiangzhou Huang and Xiangyu Xi and Wei Wang and Jingang Wang and Xunliang Cai and Shikun Zhang and Wei Ye},
  title = {{Autoformalizer with Tool Feedback}},
  year = {2025},
  eprint = {2510.06857},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.06857},
}
Figure