Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation

> Paper: *Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation* > Authors: Joshua Ong Jun Leang et al. (Imperial College London / Edinburgh / NTU / MBZUAI) > Link: https://arxiv.org/abs/2606.12594

Headline Results: 167x Fewer Parameters, Better Performance

| Model | Parameters | MiniF2F-Test pass@32 | Relative size | |-------|-----------|---------------------|---------------| | DeepSeek-Prover-V2 | 671B | 82.4% | baseline | | Pythagoras-Prover | 4B | 86.1% | 1/167 | | Pythagoras-Prover | 32B | 93.0% | 1/21 |

The 4B model outperforms the 671B DeepSeek-Prover-V2, and the 32B version sets an open-source SOTA of 93.0% without inference-time self-correction.

Why Formal Theorem Proving Is Hard

Formal theorem proving means writing proofs in machine-verifiable languages like Lean, Coq, or Isabelle, where every step must compile and check strictly. Compared with natural-language proofs, it is 100% reliable once verified, but it suffers from:

1. Data scarcity — very few humans write Lean proofs 2. Extremely long reasoning chains — hundreds of tactic steps per proof 3. Exploding training costs — long sequences plus scarce data

This is why prior state-of-the-art models relied on massive parameter counts.

The Three Pillars

1. Layered Seed Corpus

Training data is stratified by difficulty:

• Easy: auto-formalised public math datasets (e.g., NL-Lean, Mathlib) — short, basic proofs
• Medium: rubric-guided distillation from the Easy layer, generating simplified variants targeting Lean-specific errors
• Hard: competition-level problems (IMO, Putnam) used for reinforcement learning

2. Curriculum Multi-Stage Training

• Stage 1 — SFT in curriculum order (Easy → Medium → Hard), with dynamic proof-of-thought filtering to keep only informative trajectories, and each training instance capped at 8k tokens to avoid attention dilution
• Stage 2 — RL on Hard problems, with a binary reward from Lean verification
• Stage 3 — Self-distillation on ALF-expanded corpora, supporting both autoregressive and diffusion architectures

3. ALF: Augmented Lean Formalisation

Instead of re-verifying every mutated sample with Lean, ALF expands verified corpora 2.5x using lightweight checks:

1. Take a verified Lean theorem, e.g.

\[\forall n \in \mathbb{N}, n^2 \geq n\]

2. Generate variants (variable renaming:

\[\forall m \in \mathbb{N}, m^2 \geq m\]
; restructuring:
\[\forall n \in \mathbb{N}, n \cdot n \geq n\]
; equivalent transforms:
\[\forall n \in \mathbb{N}, n^2 - n \geq 0\]
) 3. Apply lightweight syntax/type validity checks instead of full Lean compilation 4. Variants preserve the formal features (structure, complexity, proof strategy) of the original

Key insight: the model doesn't need *new* theorems — it needs *different expressions of the same theorem* to avoid overfitting to surface forms.

Diffusion-Based Theorem Proving

The paper introduces the first diffusion-based theorem-proving model (4B parameters):

• Input: Lean goal state → Output: complete tactic sequence
• Inference: iterative refinement — a rough proof draft is generated, then progressively improved, allowing revision of earlier steps and parallel generation

Results: 2.58x faster generation than autoregressive decoding, with comparable accuracy at equal parameter count — especially advantageous for long proofs.

This opens a new accuracy-efficiency trade-off: diffusion for speed, autoregressive (or more diffusion iterations) for maximum accuracy.

Experimental Results

MiniF2F-Test

| Model | Parameters | pass@32 | |-------|-----------|---------| | DeepSeek-Prover-V2 | 671B | 82.4% | | Goedel-Prover-V2-32B | 32B | ~85% | | Pythagoras-Prover-4B | 4B | 86.1% | | Pythagoras-Prover-32B | 32B | 93.0% |

PutnamBench

Pythagoras-Prover-32B solves 93 of 672 problems — the best open-source result.

MiniF2F-ALF (contamination-sensitive benchmark)

Using ALF mutations of the MiniF2F test set, all existing models drop in performance — evidence of overfitting to surface forms. Pythagoras-Prover-32B remains the strongest, and the 4B model matches the previous best, Goedel-Prover-V2-32B.

Why Can a Small Model Win?

• Data quality > model scale: the 671B model may be data-starved in this scarce-data domain, while ALF extends effective training signal 2.5x
• Context efficiency: the 8k-token cap keeps attention efficient — dense short sequences beat diluted long ones
• Diffusion architecture: iterative refinement may be inherently better suited to proof generation

Limitations

• Validated only on Lean; transfer to Coq/Isabelle requires more work
• Difficulty ceiling: only 93/672 (13.8%) on PutnamBench
• Diffusion proving is still at proof-of-concept stage (4B only, trade-offs not fully characterized)

Takeaways

• In data-scarce domains, augmentation methods like ALF may beat simply scaling parameters
• Curriculum learning by difficulty is critical for long-sequence, complex reasoning tasks
• Diffusion architectures may benefit any task amenable to iterative refinement — text, code, and proofs alike

> Core conclusion: Pythagoras-Prover shows that smart data strategy and efficient training can close — and even reverse — a 167x parameter gap in formal theorem proving.

References

1. Ong, J. J. L., et al. (2026). *Pythagoras-Prover: Advancing Efficient Formal Proving via Augmented Lean Formalisation*. arXiv:2606.12594. 2. Xin, H., et al. (2024). DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning. *arXiv preprint*. 3. Goedel-Prover Team. (2025). Goedel-Prover-V2: A Scalable Neuro-Symbolic Theorem Prover. *arXiv preprint*. 4. Jiang, A. Q., et al. (2022). Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers. *NeurIPS 2022*. 5. Yang, K., et al. (2023). LeanDojo: Theorem Proving with Retrieval-Augmented Language Models. *NeurIPS 2023*.