PaperFormalizer: Author-in-the-Loop Autoformalization of Research Papers in Lean

Jiyuan Tan, Vasilis Syrgkanis, Jose Blanchet

TL;DR

PaperFormalizer is an agentic pipeline that helps researchers transform their LaTeX-based papers into Lean-verified mathematical proofs. By keeping the author "in the loop" to resolve natural language ambiguities and approve error repairs, it makes rigorous formal verification a practical part of the scientific writing process.

What Problem Does This Paper Address?

Even the most meticulously proofread research papers often contain "small gaps"—overlooked boundary cases, missing regularity conditions, or implicit assumptions. While proof assistants like Lean can catch these errors, manual formalization is incredibly time-consuming, often taking weeks for a single paper.

Current AI-driven "autoformalization" tools face two major hurdles:

  1. Ambiguity: Natural language is imprecise. A machine might formalize a statement in a way the author never intended.
  2. Repair Choice: When a statement is found to be false, there are many ways to fix it. A machine cannot know which "repair" preserves the author's core scientific contribution.

PaperFormalizer addresses these by treating formalization as a collaborative process between an AI agent and the human author.

Methods

The PaperFormalizer pipeline consists of two main stages, as illustrated in the workflow below:

The two-stage workflow of PaperFormalizer: Natural language review followed by autoformalization.

Stage I: Natural Language Review

  • Transcription: An agent converts the LaTeX source into a dependency graph of definitions, assumptions, and theorems.
  • Review: A reviewer agent checks each node for faithfulness to the text, ambiguity, and mathematical correctness. If it finds a potential error, a "fixer" agent proposes a minimal repair.

Stage II: Autoformalization

  • Planning & Scaffolding: The system decides which parts to reuse from existing libraries (like Mathlib) and which to formalize from scratch. It then writes the Lean "scaffolding" (the statements of the theorems).
  • Proof Filling: AI "workers" attempt to fill in the proofs, querying Lean's language server to ensure every step is valid.
  • Linking: Finally, the system links every Lean declaration back to the specific sentence in the paper it formalizes, allowing the author to verify the translation.
The PaperFormalizer viewer showing how Lean code hypotheses are linked directly to the paper's text.

The system runs inside coding agents like Claude Code or Codex and provides a local web interface for the author to review findings and approve changes.

Datasets

The authors evaluated PaperFormalizer on three of their own theoretical papers in fields like statistics and machine learning:

  1. A paper on Treatment Effects on Extreme Events.
  2. A paper on Optimal Transport.
  3. A paper on Neural Causal Models.

Key Insights and Ideas

  • Author-in-the-Loop is Essential: The system is designed to stop and ask the author whenever a statement has multiple readings or a repair might change a main result. This ensures the final verified version actually reflects the author's intent.
  • The "Personal Toolkit": As an author formalizes more papers, they build a library of reusable "Foundations" and "Tricks." This creates a feedback loop that makes each subsequent paper easier and cheaper to verify.
  • Formalization as a Debugger: The process isn't just about getting a "stamp of approval"; it's a powerful way to find and fix subtle errors during the writing process itself.

Key Results

In their primary case study (a paper on extreme treatment effects), PaperFormalizer revealed that 5 out of 7 results did not hold as printed.

The PaperFormalizer UI showing a repaired identification result. The original (red) was found to be missing specific conditions, which the repair (blue) adds.

The pipeline identified the root causes—such as missing exogeneity conditions and incorrect radial limits—and proposed repairs that the authors accepted. Ultimately, all seven results were successfully verified in Lean.

In other tests:

  • Optimal Transport: All 12 results were proved exactly as printed.
  • Neural Causal Models: 20 of 29 results were proved; two non-main results were refuted (shown to be false with no simple repair).

Limitations and Future Work

  • Human Time: While much faster than manual formalization, the author still needs to spend time reviewing AI findings and resolving ambiguities.
  • Cost: Running high-reasoning LLMs (like GPT-4 or Claude 3 Opus) through many iterations can be expensive.
  • Future Directions: The authors hope PaperFormalizer becomes a routine part of the research workflow, allowing scientists to share "Lean-verified" papers that others can build upon with absolute confidence in the underlying mathematics.

Summary updated

Abstract

Even a carefully proofread paper can leave small gaps in its proofs or its stated assumptions. We introduce PaperFormalizer, a Lean-based agentic pipeline that helps authors find and close such gaps in their own papers and turn each paper into a Lean-verified version. Starting from the LaTeX source, PaperFormalizer transcribes the paper into a dependency graph of statements, audits each statement in natural language for faithfulness, ambiguity and correctness, and then formalizes and proves the statements in Lean against Mathlib, optional domain libraries and the author's personal toolkit. Only the author knows what a statement is meant to say and how a formalization gap in a result is best fixed. The pipeline therefore stops and asks the author whenever a statement admits more than one reading, presents every repair, in natural language and TeX symbols, as a proposal that the author can accept, decline or redirect, and leaves to the author every decision that could potentially change what the paper claims. The output is an annotated and revised version of the paper, a report of the changes, and Lean code in which each declaration is linked to the statement it formalizes. The author's toolkit grows as more papers are formalized, and authors can share their verified papers so that researchers who build on them can reuse their Lean code. By making verified papers practical to produce, PaperFormalizer aims to make research that rests on mathematical proofs more robust. PaperFormalizer runs inside a coding agent (Claude Code or Codex) and is available at https://github.com/Jiyuan-Tan/PaperFormalizer. Papers with Lean code shared by users are available at https://paperformalizer.org.
BibTeX
@misc{tan2026paperformalizer,
  author = {Jiyuan Tan and Vasilis Syrgkanis and Jose Blanchet},
  title = {{PaperFormalizer: Author-in-the-Loop Autoformalization of Research Papers in Lean}},
  year = {2026},
  eprint = {2610.31810},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.31810},
}
Figure