Leanstral 1.5: Mistral's Math Theorem-Proving LLM Generates Mechanically Verified Lean 4 Proofs

Forum topic · 小凯 · 2026-07-05

Summary

Mistral AI released Leanstral 1.5 on June 30, 2026, a 119B-parameter Mixture-of-Experts model (6.5B active, 128 experts, 256K context, Apache 2.0) purpose-built for Lean 4 formal proofs and automated theorem proving. Unlike general LLMs that probabilistically guess answers, Leanstral emits proof steps that the Lean 4 type checker mechanically validates, eliminating hallucination by construction. Trained in three stages (mid-training, supervised fine-tuning on expert demonstrations, and CISPO reinforcement learning in multiturn and code-agent environments), it saturates miniF2F at 100%, solves 587/672 PutnamBench problems (87.3%), and achieves 34% on FATE-X. Test-time scaling is dramatic: 44 problems at 50K tokens, 587 at 4M tokens, at roughly $4 per proof, 15–75x cheaper than Seed-Prover 1.5 or Aleph Prover. Beyond math, it verified real code (an AVL-tree O(log n) bound) and auto-discovered 11 bugs across 57 open-source Rust repositories via Aeneas translation, including 5 previously unreported issues. Limits include Lean-4-only support and translation-layer overhead.

Overview

Source: Mistral AI Model: Leanstral 1.5 Focus: Lean 4 formal proofs and automated theorem proving Architecture: 119B MoE (6.5B active, 128 experts), 256K context window Modalities: text + image input; text output License: Apache 2.0 Release: 2026-06-30

Why "Verifiable Proofs" Matter

Traditional LLMs (GPT-4, Claude, etc.) produce probabilistic text: a likely-looking answer that may hallucinate. Lean 4 is a proof assistant — a strict, rule-based checker that validates every proof step mechanically rather than by intuition. Leanstral 1.5 combines LLM generative ability with Lean 4's verification loop: every emitted proof step must pass the Lean compiler's type checker. Either the proof succeeds or it fails; nothing is "pretend-correct." This collapses hallucination inside the Lean framework.

| Dimension | Traditional LLM | Leanstral 1.5 (Lean 4) | |---|---|---| | Output | Probabilistic text | Mechanically checked proof | | Correctness | "Probably right" | "Right if Lean accepts" | | Error mode | Confident hallucinations | Verified or rejected |

Architecture

Total parameters: 119B
Active per token: 6.5B (4 of 128 experts)
Context window: 256K tokens (critical for long proof chains)
Router picks the best 4 experts per token, balancing capacity and compute

MoE suits theorem proving because algebra, geometry, logic, set theory, and number theory benefit from different "specialists," analogous to doctors rather than a generalist.

Training Pipeline

1. Mid-training: continued pretraining on math and code to internalize Lean 4 syntax, mathematical concepts, and proof patterns. 2. Supervised Fine-Tuning (SFT): learning from human-authored Lean 4 proofs. 3. Reinforcement Learning (CISPO): trial-and-error in two environments.

*Multiturn*: state a theorem → attempt proof → submit to Lean → read compiler feedback (success or error line) → revise → repeat until success or budget exhaustion.
*Code Agent*: real filesystem — edit files, run bash, query the Lean language server for type and error info, factor out helper lemmas, and use context compaction to handle proofs longer than the 256K window.

Benchmark Results

| Benchmark | Result | |---|---| | miniF2F | 100% saturation | | PutnamBench | 587/672 (87.3%) | | FATE-H | 87% (SOTA) | | FATE-X | 34% (SOTA) |

Test-Time Scaling

| Token budget per problem | PutnamBench solved | |---|---| | 50K | 44 | | 200K | 244 | | 1M | 493 | | 4M | 587 |

Cost ≈ $4 per proof (4M tokens), compared with Seed-Prover 1.5 (high) ≈ $300/proof and Aleph Prover ≈ $54–68/proof — a 15x–75x reduction.

Real-World Code Verification

AVL-tree complexity proof: structural induction with a TimeM monad, consuming 2.7M tokens across 22 context compactions, establishing a tight ~48-step-per-unit-height upper bound on insert/delete.

Bug discovery across 57 open-source Rust repositories (via Aeneas translation into Lean 4):

47 properties flagged as violated
11 real bugs found
5 previously unreported on GitHub

Example: datrs/varinteger's zigzag-decoding signed function overflows when the input is Std.U64.MAX, since (value + 1) wraps. This crashes in debug builds and silently corrupts data in release builds.

Limitations

Lean 4 only; not transferable to Coq, Isabelle, or TLA+ without retraining.
Verification of Rust/Python requires Aeneas-style translation, which can introduce its own errors.
At ~$4/proof, validating an entire Linux kernel remains prohibitively expensive.
Lean guarantees adherence to the spec, but a flawed spec still produces a "verified" but incorrect result (garbage-in, garbage-out).

Future Directions

Multi-assistant support (Coq, Isabelle, TLA+)
Automatic specification inference so users don't hand-write what to verify
CI/CD integration: formal verification on every commit
Education: producing complete, verifiable proofs to help learners see not just the answer but the rigorous path to it

One-Sentence Takeaway

Leanstral 1.5 trades probabilistic guessing for mechanically checked proofs: a 119B/6.5B-active MoE with 256K context, trained mid-training → SFT → CISPO RL, saturating miniF2F and hitting 587/672 on PutnamBench at ~1/15 the cost of competitors, while already discovering unreported bugs in real open-source code.

References

Mistral AI announcement (2026-06-30): https://mistral.ai/news/leanstral-1.5
Hugging Face model card: https://huggingface.co/mistralai/Leanstral-1.5-119B-A6B
MarkTechPost review (2026-07-03): https://www.marktechpost.com/2026/07/03/mistral-ai-releases-leanstral-1-5
Kompozy review (2026-05-21): https://kompozy.io/reviews/leanstral-1-5