Language models that write formal proofs, meaning machine-checkable programs in a proof assistant such as Lean, are among the most expensive models to run. They typically need to sample tens to hundreds of candidate proofs per problem before one is accepted. Compressing their weights to 3 or 2 bits would make each sample much cheaper. The obstacle is that low-bit compression, called quantization, damages exactly the capabilities these models need: multi-step logic and precise symbolic manipulation break down first, and the damage grows with problem difficulty.
This proposal starts from a plain observation. A formal prover comes with a perfect grader, the Lean kernel, which accepts or rejects a proof with no noise. We propose to repair a quantized prover with expert iteration. In this procedure the model samples proofs, the kernel keeps the valid ones, and the model is fine-tuned on its own verified proofs, repeating for several rounds. The quantized model therefore heals itself using a signal that needs no full-precision teacher and no language-model judge. Because the model being sampled is the cheap quantized one, every round of the loop also costs less than the full-precision loop it replaces.
Core Claim
Recovery of quantized reasoning has so far been done off-policy and against soft targets. Quantization Meets Reasoning pairs a 4-bit model's failed natural-language solutions with a full-precision model's correct ones and runs preference optimization on a fixed set of 545 examples. That lifts Llama-3.2-3B on MATH-500 from 31.4% to 38.2%, but it relies on LLM judges to classify errors. Towards Quantization-Aware Training for Ultra-Low-Bit Reasoning LLMs trains a 2-bit student against an FP16 teacher's token probabilities on the fixed OpenThoughts corpus.
We claim that for formal proving, a different data source is more efficient per training token. The best repair data is proofs the quantized model itself produced and the kernel accepted, concentrated on the theorems it lost relative to its full-precision parent. The mechanism we hypothesize is simple. Quantization damage is idiosyncratic to each compressed network. A teacher's proofs may route through reasoning the compressed model can no longer express. The model's own verified proofs, by contrast, are proofs the compressed weights demonstrably can reach, and fine-tuning sharpens the probability of those paths.
This hypothesis could be wrong. The central experiment is built so that a simpler explanation, "any verified Lean proofs will do," would show up as a tie.
Research Program
The starting point is an existing open whole-proof prover. We use DeepSeek-Prover-V1.5, a 7B model trained with kernel-feedback reinforcement learning and tree search. As a second base we use Goedel-Prover, itself a product of expert iteration over Lean Workbook and Numina. We choose whole-proof generators because they make sampling with a fixed number of attempts per problem (pass@K) straightforward.
We quantize each base at three settings:
- 4-bit weights, 16-bit activations. Previously reported as near-lossless on natural-language reasoning by Quantization Hurts Reasoning?.
- 3-bit weights. The same study's "danger zone," with drops of 3% or more.
- About 2 bits. Uses incoherence-processed or codebook quantizers such as QuIP# and AQLM from the Vahe1994/AQLM codebase. Plain scalar methods are reported to fail completely below 3 bits (The Performance Cliff in Ultra Low Bit Quantization).
The first deliverable is a map of how quantization damages formal proving, which no paper in the file reports. For each theorem we record whether the full-precision and quantized models solve it. This defines a set of "lost theorems." We also record how search behavior changes: proof length, tactic mix, compile-error types, and time to first success. This speaks directly to the unresolved disagreement in the literature about whether low-bit models ramble. Quantization Hurts Reasoning? reports stable output lengths, while the QAT paper reports "redundant thinking" at 2 bits.
The second deliverable is the repair loop. We freeze the quantized weights and train only low-rank adapters and, where the quantizer allows, the continuous scale parameters. This keeps the deployed artifact low-bit. Each round has three steps:
- The quantized model samples K proofs per statement over a Lean Workbook training pool with miniF2F and ProofNet removed.
- The Lean kernel filters the samples.
- The adapters are fine-tuned on verified proofs.
Two design choices make this more than off-the-shelf expert iteration.
The first is lost-theorem weighting. Verified proofs of theorems the full-precision model solves but the quantized model rarely solves are upweighted. The kernel lets us identify these theorems exactly, with no judge.
The second is shortcut monitoring. Goedel-Prover reports that reinforcement learning on proofs drifts toward automation tactics such as simp_all and try. A quantized model under repair may be even more tempted by such tactics. We track the tactic distribution each round, and we deduplicate proofs that differ only in automation padding.
At 2 bits, if adapter training cannot close the gap, we swap in PV-Tuning's discrete-assignment updates as the optimizer. PV-Tuning reaches 5.71 WikiText-2 perplexity on 2-bit Llama-2 7B, versus 6.19 for QuIP#, but at roughly 1.5× fine-tuning cost. It has never been driven by verified-proof data.
Baselines
The do-nothing reference is the quantized prover as delivered by post-training quantization, calibrated on Lean and Mathlib text rather than generic text. Math calibration is itself a known fix: reasoning calibration improved GPTQ by nearly 10% in Quantization Hurts Reasoning?. The unquantized parent defines the ceiling.
For those parents, the file reports the following on miniF2F:
- Goedel-Prover-SFT: 57.6% at Pass@32.
- DeepSeek-Prover-V1.5: 63.5% on miniF2F-test at its reported budget, and 25.3% on ProofNet-test.
We reproduce these numbers ourselves under our sampling budget before quantizing. For quantized formal provers, no reported result was found on Lacuna.
The strongest existing competitor is teacher-guided repair: token-level distillation from the FP16 prover into the quantized model, following the reward-rectification recipe of Towards Quantization-Aware Training for Ultra-Low-Bit Reasoning LLMs. That recipe reports a 50.45% average improvement over 2-bit PTQ on natural-language benchmarks. We give it the same statements, the same adapter parameterization, the same training tokens, and the same tuning sweep, with the teacher's outputs on the training pool as its data. We also run the preference-optimization recipe of Quantization Meets Reasoning, with judge-classified failures replaced by kernel-rejected quantized proofs paired with kernel-accepted FP16 proofs, at the same budget.
The key ablation replaces the quantized model's own verified proofs with the FP16 model's verified proofs of the same theorems, matched in count and token length. This isolates the on-policy ingredient. A second ablation removes lost-theorem weighting.
Beating the competitors means one thing: at matched training tokens, higher miniF2F-test pass@32 at 3 and 2 bits, with ProofNet no worse. We treat any gain below the run-to-run spread across three seeds as a tie.
Why not just…?
Why not just quantize a trained prover to 4 bits and stop? At 4 bits this may be enough, and we will report it honestly if the loss is within noise. The evidence points the other way for hard problems and lower bit widths. Quantization Hurts Reasoning? finds AIME losses up to four times GSM8K losses. Quantization Meets Reasoning sees 4-bit drops above 30% on a small model. Our damage map settles the question per bit width.
Why not just distill from the full-precision prover? This is the real threat. Distillation is cheap, needs no search, and already works on informal reasoning. Our argument is that a compressed network cannot match the teacher everywhere, and verified self-generated proofs focus its limited capacity on reachable proofs. The matched distillation baseline tests this directly. If distillation ties or wins at 3 bits, the proposal's remaining claim narrows to the compute argument below and to the 2-bit regime.
Why not just fine-tune on the FP16 model's verified proofs, which expert-iteration pipelines already produce? This is the on-policy ablation. If it matches self-generated data, the lesson is that "verified Lean data repairs quantization." That is still useful but less novel, and we would report it as the finding.
Why not just use a smaller model at full precision? Quantization Hurts Reasoning? reports that a 3-bit 32B model beats a 16-bit 7B model. PV-Tuning reports that a 2-bit 13B model beats any 7B model at equal memory. We test this in formal proving with a memory-matched comparison.
Why not just run GRPO-style reinforcement learning on the quantized model? Goedel-Prover reports shortcut overfitting under RL on proofs. We include a GRPO arm at matched compute as a secondary baseline and compare tactic distributions across arms.
Proposed Experiments
Experiment 1: on-policy versus off-policy repair. This is the decisive check. At 3 and 2 bits, and at equal training tokens, we compare three data sources: the quantized model's verified proofs, the FP16 model's verified proofs, and FP16 distillation. We measure pass@32 on miniF2F-test and ProofNet-test, and recovery on the lost-theorem set. The prediction is that self-generated data recovers more lost theorems per token. A null result reframes the paper as described in the previous section.
Experiment 2: compute-matched frontier. We plot miniF2F-test solve rate against total GPU-hours, counting both loop sampling and evaluation sampling. We compare the repaired 3-bit prover, the FP16 prover, and the memory-matched smaller FP16 model. The claim succeeds if the repaired low-bit prover dominates at some budget. The rationale is that it can afford more samples per GPU-hour. We also report kernel CPU time, which quantization does not reduce.
Experiment 3: search dynamics. Across bit widths and repair rounds, we measure proof length, tactic entropy, the share of automation-only proofs, and timeouts. This tests whether low-bit provers show redundant exploration, and whether the loop removes it or entrenches shortcuts.
Experiment 4: transfer. We evaluate on PutnamBench and ProofNet, which are out of distribution for the Lean Workbook training pool. Goedel-Prover reports a negative correlation between ProofNet and competition benchmarks. A repair that raises miniF2F while lowering ProofNet counts against the method, and we report it that way.
References
- Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
- Towards Quantization-Aware Training for Ultra-Low-Bit Reasoning LLMs
- DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Search
- Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Proving
- Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
- QuIP$\#$: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks
- Cherry on Top: Parameter Heterogeneity and Quantization in Large Language Models
- PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression