TL;DR
The "Bitter Lesson" of AI—that general methods scaling with compute eventually outperform human-engineered domain knowledge—has arrived for automated theorem proving. The authors introduce h1, a simple agentic loop that eschews complex, human-designed workflows in favor of a streamlined interface to the Lean 4 proof assistant and test-time scaling. h1 achieves state-of-the-art results on major benchmarks and settles eight previously open mathematical problems, demonstrating that a single, persistent attempt can often outperform specialized multi-agent systems.
What Problem Does This Paper Address?
Formal theorem proving is the "hard mode" of mathematics, requiring proofs to be written in code (like Lean 4) that can be mechanically verified. As frontier Large Language Models (LLMs) have improved, researchers have built increasingly complex "harnesses" around them—systems that enforce specific planning stages, decompose problems into sub-lemmas, or orchestrate multiple specialized agents.
The authors ask a fundamental question: Are these complex, human-designed workflows actually necessary? Or are their gains simply a byproduct of using stronger models and more compute? By testing whether a simple, unstructured loop can match or beat these sophisticated systems, the paper explores whether the "Bitter Lesson" applies to the design of theorem-proving agents.
Methods
The authors propose h1, a "simple harness" that provides a model with an efficient interface to Lean 4 without imposing a specific search or decomposition structure.
The h1 loop consists of three main stages:
- Proof Search (ReAct): The model uses a multi-turn "Reasoning and Acting" loop to edit a persistent working file, call search tools (like Mathlib search), and receive immediate compiler feedback.
- Validation: An external validator (SafeVerify) checks the final submission for correctness and ensures no "cheating" (like using
sorryplaceholders) occurred. - Reflection: If validation fails, a separate "Reflection LLM" compacts the entire interaction transcript into a concise summary of verified progress and lessons learned. This summary becomes the "memory" for the next iteration, preventing the model from repeating mistakes.
Crucially, h1 focuses on interface efficiency. It uses targeted file edits rather than full-proof regeneration and leverages heavy caching to make long-horizon reasoning affordable.
Datasets
The authors evaluated h1 across a wide range of benchmarks:
- PutnamBench: 672 problems from the prestigious Putnam undergraduate competition.
- FATE-X: 100 algebra problems at the PhD qualifying-exam level.
- FormalConjectures100-Solved: A subset of research-level conjectures with known solutions.
- IMO 2026: Problems from the International Mathematical Olympiad.
- Open Problems: 236 targets from the Formal Conjectures dataset that were previously unsolved.
Key Insights and Ideas
- The Bitter Lesson Reaffirmed: Simple scaling Pareto-dominates specialized systems. When using the same underlying model, h1 often achieves higher solve rates at lower costs than complex systems like Goedel-Architect or AlphaProof Nexus.
- Interface over Workflow: The authors found that how the model interacts with the environment matters more than the structure of its thoughts. Features like persistent workspaces, immediate compiler feedback, and efficient library search are the primary drivers of success.
- Depth beats Breadth: In theorem proving, a single long, persistent attempt (pass@1 with high budget) is often more efficient than many independent short attempts (pass@k). This challenges the common practice of "repeated sampling" as the primary way to scale test-time compute.
- Reflection Prevents "Context Rot": Without the reflection step to compact history, long agentic runs eventually suffer from performance degradation as the context window fills with irrelevant trial-and-error.
Key Results
- State-of-the-Art Performance: Using GPT-5.5, h1 solved 670 out of 672 problems on PutnamBench. On FATE-X, it solved 90/100 problems, surpassing specialized provers.
- Settling Open Problems: h1 successfully proved eight open problems in the Formal Conjectures dataset, including variants of Erdős conjectures and OEIS sequences.
- Efficiency Gains: On a 100-problem PutnamBench subset, h1 with Gemini 3 Flash significantly outperformed AxProverBase (a more complex framework) by solving 69 problems compared to 18 at a matched $20 budget.
Limitations and Future Work
- Single-File Constraint: h1 currently operates on a single Lean file. Proving massive theorems (like Fermat’s Last Theorem) that require managing thousands of lines across multiple files will likely require new forms of dependency management.
- Black-Box Models: The authors acknowledge that because they use proprietary frontier models, it is difficult to perfectly account for potential data leakage during the models' pre-training.
- Human-Designed Tools: While the workflow is simple, the tools (like search engines for Mathlib) are still human-designed. The authors suggest that in the future, models might even design and deploy their own custom harnesses for specific problems.
Summary updated