TL;DR
Researchers successfully ported CompCert, the industry-standard verified C compiler, from Rocq to Lean in just six weeks using a team of parallel LLM agents directed by a single human expert. The resulting CompCert4Lean not only maintains full formal correctness but also proves to be a superior environment for AI-driven optimizations, achieving an 11.7% reduction in program runtime compared to only 2.3% for the original Rocq version.
What Problem Does This Paper Address?
Formal verification—mathematically proving that software is bug-free—is incredibly labor-intensive. CompCert, a landmark in this field, took roughly three person-years to build its first version and contains nearly 150,000 lines of code in the Rocq (formerly Coq) proof assistant.
However, the research community is increasingly moving toward Lean, a modern proof assistant with a massive mathematical library (Mathlib) and better support for AI agents. The "Rocq-to-Lean" gap means years of verified infrastructure are currently inaccessible to the latest AI tools. Manually porting such a massive project would normally take years; this paper explores whether LLMs can collapse that timeline while maintaining the rigorous standards of formal proof.
Methods
The authors developed a novel LLM Migration Workflow designed to handle the "dense web of dependencies" inherent in verified software. Instead of a naive file-by-file translation, they used a multi-stage approach:
- Dependency Partitioning: The project was cut into 166 parallel units based on their dependency graph.
- Definitions-First Migration: Agents first translated definitions and theorem statements. Crucially, they deferred the actual proofs at this stage to allow downstream work to proceed against stable interfaces.
- Massive Parallel Review: Independent LLM agents audited the translated definitions against the Rocq source. This caught critical errors (like theorems accidentally being reduced to "True") before they could propagate.
- Parallel Proof Reconstruction: Once the "skeleton" was verified, agents reconstructed all 4,908 proofs in parallel.
- Human Coordination: A single human expert acted as a "manager," focusing 88.8% of their interactions on workflow and coordination rather than writing code or solving technical proofs.
Datasets
The authors evaluated the compiler's correctness and performance using:
- CompCert Small-Tests: A suite of 139 programs covering language regressions and numerical kernels.
- SPEC CPU2026: Six large, real-world C programs (e.g., SQLite, Zstd) used to measure performance.
- Differential Testing Corpus: A massive set of 22,062 test records, including random programs generated by Csmith and YARPGen, to ensure the Lean port behaved identically to the original Rocq version.
Key Insights and Ideas
- The "True" Trap: The first audit round caught 1,091 theorem statements that had been incorrectly translated such that they were vacuously true (e.g.,
1=1). Without the "definitions-first" audit, these errors would have been buried under thousands of lines of "successful" but meaningless proofs. - Lean as an AI Playground: The study found that LLMs (specifically GPT-6-astra) were far more effective at optimizing code in Lean than in Rocq. The agent produced 106 successful compiler variants in Lean compared to only 40 in Rocq under the same time budget.
- Coordination over Coding: The workflow shifted the human's role from a "proof engineer" to a "workflow orchestrator." The expert only provided three instances of actual technical direction during the entire six-week process.
Key Results
- Speed: Ported 149,038 lines of Rocq to 391,963 lines of Lean in six weeks.
- Completeness: Discharged all 4,908 proof obligations without using project-specific axioms.
- Fidelity: In differential testing, the Lean version produced byte-identical intermediate representation (RTL) for over 18,000 programs compared to the original CompCert.
- Optimization Gains: Using a 24-hour budget, an LLM agent improved the Lean compiler's generated code performance by 11.7%, significantly outperforming the 2.3% improvement it managed on the Rocq version.
Limitations and Future Work
- Compilation Overhead: The Lean port currently compiles code about 2x slower than the original OCaml-based CompCert.
- Provisional Benchmarks: The SPEC performance gains are considered "provisional" due to differences in measurement protocols across environments.
- Mechanized Equivalence: While the port is verified internally, there is no "mechanized equivalence theorem" that mathematically links the original Rocq semantics to the new Lean semantics; the link currently relies on extensive auditing and testing.
- Future Directions: The authors suggest this workflow could be used to migrate other massive Rocq-based tools, such as the Iris separation logic framework or verified operating system kernels like CertiKOS.
Summary updated