TL;DR
This position paper argues that the field of formal reasoning is being held back by "incomplete" benchmarks and a culture of secrecy regarding training data and code. The authors advocate for a new standard: benchmarks must include formal/informal versions of both problem statements and proofs, coupled with an open-science approach to accelerate AI progress in automated theorem proving.
What Problem Does This Paper Address?
The field of formal reasoning—where AI models attempt to solve complex math problems using machine-verifiable languages like Lean—is currently a "frontier" of machine learning. However, the authors identify several critical bottlenecks:
- Data Scarcity: High-quality pairs of informal (natural language) and formal (code) mathematical statements are rare.
- Evaluation Inflation: Researchers are increasingly using LLMs to "judge" the accuracy of other LLMs. The authors show this leads to wildly inflated success rates (e.g., a reported 97% accuracy that is actually 67% when checked by a human expert).
- Benchmark Errors: Even gold-standard benchmarks like miniF2F contain unprovable theorems and errors that take years for the community to discover.
- The "Black Box" Culture: To avoid "data contamination," many teams keep their formal proofs and training data private. This prevents other researchers from understanding why a model failed or how to improve upon it.
Methods
As a position paper, this work does not introduce a new algorithm but rather provides a critical framework for the community. The authors:
- Reproduce and Audit: They reproduced the evaluation pipeline of Herald (a state-of-the-art autoformalization model) to demonstrate the discrepancy between automated LLM judging and human expert verification.
- Comparative Analysis: They compared the outputs of current models (like Kimina Prover) against manually written formal proofs to show how "short" generated proofs often fail to capture the complexity of the actual problem.
- Proposed Standard: They define a "Complete Benchmark" as one containing four essential components:
- Informal problem statements.
- Formal problem statements.
- Informal proofs.
- Formal proofs.
Datasets
The paper discusses several pivotal benchmarks in the field:
- miniF2F: The standard benchmark for Olympiad-level math. The authors highlight that it still contains discrepancies between its formal and informal statements.
- Putnam Bench: A newer, university-level benchmark. The authors criticize its policy of asking users to keep generated proofs private, which they argue creates a barrier to entry.
- Mathlib: The massive library of Lean theorems, which lacks elaborate natural language descriptions for many entries.
- ImageNet: Used as a historical parallel to argue that open access to labels and data is what drives a field forward.
Key Insights and Ideas
- LLMs are Bad Referees: The study found that LLM-based evaluation pipelines for autoformalization are unreliable. Models can produce "plausible-looking" formal statements that are semantically wrong, which an LLM judge might miss but a Lean compiler or human expert would catch immediately.
- The Transparency Tax: By not releasing fine-tuning scripts or search algorithms (like those used in BFS-Prover), the community forces researchers into "parallel redundant work," where everyone spends months trying to replicate the same baseline.
- Proof Length as a Metric: Accuracy is a "binary" and often misleading metric. The authors suggest reporting proof lengths and the use of automatic solvers (tactics like
linarith). If a model "solves" a problem in 3 lines that requires 1,200 lines of manual Lean code, it likely simplified the problem out of existence rather than solving it. - Lean is Manual Labor: The authors argue that writing formal proofs is a form of "manual labor" similar to writing complex software. This effort should be credited and shared, rather than hidden to prevent "contamination."
Key Results
- The "Herald" Discrepancy: The authors found that the Herald model, which claimed 97% accuracy on miniF2F via an automated LLM judge, actually had a true accuracy of 67% when manually verified by a Lean expert.
- Benchmark Decay: Four years after the release of miniF2F, new errors and unprovable theorems are still being found (5 reported as recently as April 2025). This suggests that without ground-truth formal proofs provided by benchmark creators, errors persist indefinitely.
- Model Failure Analysis: In one case (IMO 1985 P6), a model generated a 3-line proof for a problem that requires over 1,200 lines of formal code, illustrating a total lack of correlation between model output and the actual mathematical requirements.
Limitations and Future Work
- The Contamination Risk: The authors acknowledge the primary counter-argument: if you release all proofs, LLMs will simply memorize them. However, they argue that "memorization" is a step toward "reasoning" and that the benefits of transparency outweigh the risks of contamination.
- Human-in-the-loop: They admit that, for now, we cannot fully automate the evaluation of formal reasoning. Future work should focus on building more reliable automated pipelines that don't rely on "black box" LLM judges.
- Call to Action: The authors urge the community to treat formal reasoning like the Computer Vision community treats ImageNet: release the code, release the data, and credit the manual effort required to build high-quality datasets.
Summary updated