English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Vero: Can AI Agents Build Formally Verified Software Repositories? — Benchmark Analysis

Forum topic · 小凯 · 2026-08-14

Summary

Vero (arXiv:2608.13522) is the first benchmark evaluating whether AI agents can jointly generate implementations and machine-checked proofs at the repository level. Built from 43 real-world instances spanning four languages (Python, Dafny, Verus, Coq), curated into Lean 4 format, Vero provides predetermined API interfaces, manually curated formal specifications, and reference implementations. It supports two evaluation modes—proof-only and code-and-proof—plus a novel audit mechanism allowing agents to flag unsatisfiable specifications or incorrect reference implementations. Under the strongest agent configuration with Lean toolchain access, the best agent fully resolved only 27 of 43 instances (63%), and closed no specifications on the hardest repository, exposing a large gap between single-function code generation and whole-repository formal verification. The post explains why formal verification is hard for AI (technical difficulty of proof tactics, multi-module architectural complexity, and the need for creative reasoning such as introducing auxiliary lemmas) and argues that mathematical guarantees, not tests, are the path to trustworthy AI-generated code. The benchmark, curation pipeline, and evaluation framework are open-sourced at https://github.com/sunblaze-ucb/vero.

Vero: When AI Starts Proving Itself — Formal Verification and Agent Self-Awareness

*Post from zhichai.net analyzing the paper:* Vero: Can AI Agents Build Formally Verified Software Repositories? *Authors: Zhe Ye, Hantao Lou, Yuechun Sun, Peiyang Song, Zhengxu Yan, Timothe Kasriel, Qingyang Zhang, Kaiyu Yang, Soonho Kong, Jingxuan He, Dawn Song* *arXiv: 2608.13522*

Prologue: Testimony Without Evidence

Imagine a courtroom scene. The defendant: an AI coding assistant accused of generating buggy code that crashed production. Its defense: "My code passed all the tests." The prosecutor's reply cuts to the core problem: tests can only show the presence of bugs, never their absence. Current AI coding assistants can write code quickly and elegantly, but they cannot guarantee correctness. Formal verification addresses exactly this — not approximate confirmation via testing, but an absolute mathematical proof.

Chapter 1: Formal Verification — From Faith to Mathematics

Formal verification means proving mathematically that a program does what it is supposed to do. The post uses an analogy:

  • Testing is like trying 100 keys on a lock, failing each time, and declaring the lock secure — you don't know about the 101st key.
  • Formal verification is proving from the lock's internal structure that no key can open it — a mathematical statement that, once proven, is absolute.
  • Formal verification involves three parts:

    1. Specification — a formal description of what the program should do 2. Implementation — the actual code 3. Proof — a mathematical proof that the implementation satisfies the specification

    Famous successes include CompCert (a C compiler proven correct in Coq) and the seL4 microkernel (proven in Isabelle/HOL to satisfy its security specification — certain classes of vulnerabilities are mathematically impossible, not merely unlikely).

    Chapter 2: The Trust Crisis in AI Coding Assistants

  • No guarantees: generated code may look plausible and pass unit tests, yet hide bugs. A 2023 Stanford study found programmers using AI assistants produced *less* secure code, over-trusting the AI's apparent authority.
  • No global view: existing benchmarks evaluate single functions, while real systems have multi-module interfaces and implicit dependencies.
  • Spec/implementation disconnect: benchmarks usually test "given implementation, generate proof" or "given spec, generate implementation," rarely joint generation of both, matched together.
  • Chapter 3: Vero — The First Repository-Level Verified Benchmark

    Vero is the first benchmark evaluating joint generation of implementations and proofs at the repository level.

    Composition

  • 43 instances from real-world repositories
  • 4 languages: Python, Dafny, Verus, Coq
  • Domains from cryptographic protocols to distributed systems
  • All instances curated into Lean 4 repositories
  • Each instance includes predetermined API interfaces, manually curated formal specifications, and reference implementations.

    Two Evaluation Modes

    1. Proof-Only: given an implementation, generate the proof 2. Code-and-Proof: generate both implementation and proof simultaneously

    The Audit Mechanism

    Uniquely, Vero lets agents prove that a given specification is unsatisfiable or that a reference implementation is incorrect. Like a student telling the teacher the exam question itself is flawed — if correct, credit is due. This mechanism surfaced and corrected potential errors in specifications and reference code, making the benchmark more reliable.

    Chapter 4: The Sobering Result — 27/43

    With the strongest agent configuration (with Lean toolchain access):

  • The best agent fully solved only 27 of 43 instances (~63%)
  • On the hardest repository, it closed no specifications at all
  • This is Vero's value: it honestly measures how far we are from "AI building formally verified software repositories." AI handles simple, modular, well-specified tasks; complex, cross-module, deep-reasoning tasks remain out of reach.

    Chapter 5: Why Is This So Hard?

    1. Technical difficulty: understanding dependent type theory (Lean), mastering proof tactics, handling complex type systems, heavy search and backtracking — even for humans, skilled Lean provers are rare. 2. Architectural complexity: repository-level verification requires reasoning about inter-module relations, interface consistency, global constraints, circular dependencies — an order of magnitude harder than proving a sorting function correct. 3. Creative reasoning: proofs often require constructing auxiliary functions, introducing intermediate lemmas, applying advanced theorems — beyond pattern matching or brute-force search, like a Go "brilliant move" born of insight rather than pure computation.

    Chapter 6: The Metaphor — Mathematical Knights and Code Castles

    An ordinary AI coding assistant is an architect: it draws blueprints fast but never checks whether the foundation is sound. A formally-verifying AI is a mathematical knight: it builds the castle *and* provides a mathematical proof for every brick, wall, and vault. Vero is a knight's trial: build to the blueprint (specifications), prove each part safe, and — if the blueprint itself is flawed — say so (the audit mechanism). Today's strongest AI knight passes 63% of the trial.

    Chapter 7: Why It Matters — A Foundation for Trust

    > Can we trust AI-generated code?

    Today's answer is: not fully. Tests catch obvious bugs; static analysis catches some patterns; code review depends on fallible humans. Formal verification offers an absolute mathematical guarantee — assuming the proof system itself is sound, a verified program provably satisfies its specification. Vero shows this path is hard, but worth walking.

    Epilogue: The Shape of the Future

    The post sketches a timeline:

  • Short term (1–3 years): better per-function verification, more automated proof generation, human–AI collaborative workflows (AI drafts proofs, humans fill key steps)
  • Medium term (3–5 years): multi-module verification, automated specification inference, formal methods as standard practice for safety-critical software (medical, aviation, finance)
  • Long term (5–10 years): autonomous construction and verification of complex systems; "verifiable AI" becoming the norm — with AI behavior itself formally verified
The authors have released the benchmark, curation pipeline, and evaluation framework at: https://github.com/sunblaze-ucb/vero

As Feynman said:

> "The first principle is that you must not fool yourself — and you are the easiest person to fool."

Formal verification is a necessary road toward truly trustworthy AI.

Reference

Ye, Z., Lou, H., Sun, Y., Song, P., Yan, Z., Kasriel, T., Zhang, Q., Yang, K., Kong, S., He, J., & Song, D. (2026). Vero: Can AI Agents Build Formally Verified Software Repositories? arXiv preprint arXiv:2608.13522.

Tags

#formal-verification#ai-agents#benchmark#lean#software-engineering#proof-generation#ai-coding#vero

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178633485