After reading the UCL paper Patch2Vuln, a picture comes to mind: a security analyst sits in front of two monitors — on the left, yesterday's "healthy" binary; on the right, today's freshly pushed "fixed" version. Between them lies nothing but hex dumps and Ghidra decompiler output. The analyst's task is not to write an exploit, but to answer a question that seems simple yet is extremely hard: what vulnerability did this patch actually fix?
That is the core premise of Patch2Vuln. Authors Isaac David and Arthur Gervais of University College London formalize this as vulnerability reconstruction from binary patch pairs. The agent cannot see CVE pages, security advisories, or source-code patches, and cannot go online. Its only inputs are local binary evidence: ELF metadata, symbol tables, strings, Ghidra decompiled diffs, and limited local old/new behavioral validation.
1. Why is this problem worth studying?
Security patches create a peculiar time window. Shortly after a patch ships, both attackers and defenders can see the difference between the "vulnerable old version" and the "fixed new version." Prior work — such as Brumley et al.'s patch-based exploit generation — has shown that in some cases, comparing the vulnerable and patched versions is enough to synthesize a working exploit. Systems like BinXray use this diff to build vulnerability signatures for large-scale one-day vulnerability identification.
But Patch2Vuln asks a different question: can an offline LLM agent reconstruct the security meaning of a vulnerability using only local binary evidence? Not exploit generation — rather, localizing the security-relevant changed region, inferring the root-cause category (bounds check, integer overflow, null dereference, etc.), identifying the input medium, and proposing verifiable behavioral hypotheses.
> This framing matters. The paper explicitly states that a crashing PoC is a strong validation signal but not the primary metric. Many distribution patches add hard-to-trigger high-threshold integer limits, parser state guards, or defense-in-depth checks — these are still worth understanding even if fuzzing cannot easily trigger them.
2. System architecture: a candidate-centric, three-stage audit
Patch2Vuln's core is not a new binary differ, but an agentic harness that converts the output of existing tools (Ghidra + Ghidriff) into structured, vulnerability-level explanations.
The pipeline has four deterministic stages and three agent stages:
| Stage | Type | Function |
|------|------|------|
| Extract old/new packages | Deterministic | dpkg-deb -x extraction, ELF metadata collection |
| Ghidra/Ghidriff diff analysis | Deterministic | Changed-function list and decompiled snippets |
| Candidate ranking | Deterministic | Re-rank changed functions by memory-safety features (new comparisons against INT_MAX/SIZE_MAX, file-length checks, memcpy argument changes, etc.) |
| Build candidate dossiers | Deterministic | Per-function dossier (binary paths, ranking features, diff metadata, nearby strings, decompiled snippets, local call context) |
| Preliminary audit | Agent | Infer vulnerability class, security-relevant functions, validation hypotheses from static evidence |
| Validation plan | Agent | Select safe local action templates (tcpdump_pcap, expat_xmlwf, libarchive_archive, etc.); harness renders bounded inputs |
| Final audit | Agent | Revise conclusions, confidence, and candidate ranking based on observed behavioral differences |
> A key design decision: prompt construction is structure-aware. Each candidate has a fixed budget, ranking order is preserved, and the number of omitted candidates is recorded. This avoids a common failure mode where evidence is truncated mid-object, leaving the model without complete function dossiers or awareness of what was omitted.
3. Benchmark: 25 Ubuntu .deb package pairs
The evaluation covers 25 targets: 20 security update pairs plus 5 negative controls. Targets span parser libraries (Expat, libxml2, SQLite, zlib), networking/protocols (tcpdump, curl, OpenSSL, GnuTLS), archive/media (libarchive, TIFF, JPEG, WebP, Poppler), and noisy patch clusters (Ubuntu frequently bundles multiple CVE fixes into one release).
| Metric | Result | |------|------| | Security targets where the agent localized the vuln | 10 / 20 | | Security targets with correct root-cause classification | 11 / 20 | | Negative controls correctly rejected (unknown) | 5 / 5 | | Crashes / timeouts / sanitizer findings | 0 | | Behavioral differences found (non-exploit) | 2 (both tcpdump) |
The failure analysis (Figure 2) is refreshingly honest:
- 6 rank/diff misses: the ground-truth function never entered the candidate set — a binary diff coverage problem, not a model reasoning failure
- 1 context-export miss: the function was a candidate, but Ghidra context export failed
- 3 model/validation misses: the candidate reached the prompt, but agent reasoning or validation failed
4. Representative cases
tcpdump bionic (success): the ground-truth filter-file path ranked 6th, was included in the prompt, was selected for validation, and was named in the final report. Validation used an oversized sparse filter file to produce a diagnostic difference — the old version reported a short-read mismatch and a negative expected size; the new version rejected it as too large. This is the correct behavioral difference for CVE-2018-16301 — not a crash.
tcpdump xenial (failure, but interesting): a large version jump with 496 changed functions; the ground-truth function never appeared among candidates. Yet bounded local search still found a 113-byte UDP pcap behavioral difference in the BFD packet parser — the old version printed Poll, Reserved, Reserved, Reserved; the new version printed Poll, Authentication Present, Reserved, Reserved. A real parser behavior change, just not the sealed-oracle-annotated one.
Expat (static reconstruction success, no trigger): the candidate ranked 1st, was correctly classified as integer_overflow, describing size arithmetic and allocation/copy hardening. 41 bounded probes produced no old/new difference. The final report handled this correctly: static evidence supports integer-overflow hardening, but actual triggerability was not demonstrated.
libarchive (correct conservative classification): the agent localized the true function family but ultimately classified it as unknown, because bounded validation could not reproduce a behavioral difference and the patch cluster spanned multiple archive formats. 36 probes, no difference. This is the right behavior for a defensive auditing agent — not erasing strong static evidence because bounded testing failed, but not overstating memory corruption either.
Negative controls (hallucination resistance): all 5 negative controls returned unknown. Two were byte-identical controls (tcpdump, Expat); three contained real changed functions but non-security fixes (WebP with 727, OpenJPEG with 303, zlib with 20). The agent was not seduced by the "a diff exists, so there must be a vulnerability" bias.
5. Three key honest conclusions
Patch2Vuln's conclusions are not "we solved automated vulnerability analysis," but three precise honest assessments:
1. Agentic vulnerability reconstruction from binary patches is viable — not through automatic exploit generation, but by converting raw binary diffs into structured hypotheses.
2. The biggest bottleneck is not model reasoning, but binary diff coverage and candidate reachability tracking — 6 of 10 failures occurred before evidence reached the model.
3. Bounded local validation is a necessary but difficult complement — only 2 behavioral differences were found, no crashes, but those two already proved that "the changed parser paths are reachable."
> The ethics statement is also well-considered: this work is inherently dual-use, but the system is explicitly scoped to understanding and auditing — the agent has no advisories or network access during reconstruction, validation runs only on local old/new binaries inside Docker, and the reported output is vulnerability explanations and diagnostic differences, not exploit payloads.
Closing thoughts
Patch2Vuln suggests a broader observation: the most valuable application of LLM agents in security may not be replacing human hackers to write exploits, but converting low-SNR raw analysis output (hundreds of changed functions, synthetic function names, noisy decompiled snippets) into human-readable security narratives. In this sense, Patch2Vuln's 50% localization rate and 55% correct classification rate are not "not good enough" — they are an honest baseline that precisely marks the boundary of current systems and points to where the next investment should go (adjacent-version micro-diffs, better binary diff coverage, candidate reachability tracking).
That is far more valuable than papers that report only accuracy numbers without analyzing failure modes.
---
Paper metadata
| Item | Detail | |------|------| | Title | Patch2Vuln: Agentic Reconstruction of Vulnerabilities from Linux Distribution Binary Patches | | Authors | Isaac David, Arthur Gervais | | Institution | University College London | | arXiv ID | 2605.06601 | | Published | 2026-05-07 | | Link | https://arxiv.org/abs/2605.06601 | | Core contribution | Task definition and pipeline for agentic vulnerability reconstruction from Linux distribution binary patches; three-stage audit architecture; 25-case Ubuntu .deb benchmark | | Key results | 10 of 20 security targets localized, 11 root causes correctly classified; all 5 negative controls rejected; 2 behavioral differences (non-exploit) |