Kimi K3's Cybersecurity Report Card: 32% on ExploitBench, Zero ACE Across 41 V8 Vulnerabilities

A report card that is easy to misread.

The UK AI Security Institute and the US CAISI jointly tested Kimi K3. The three headline numbers getting the most media attention: Kimi K3 scored roughly 32.2% on ExploitBench, GLM-5.2 about 24.4%, and the strongest group of US frontier models about 76.2%. The gap is large. But reporting it merely as "44 points behind" misses what is actually useful in this evaluation. [1][2]

ExploitBench is not an ordinary coding test. It collects 41 Chrome V8 vulnerabilities from after 2023 and asks models to climb a realistic exploitation ladder: reproduce the crash, achieve arbitrary read/write, hijack control flow, and finally reach Arbitrary Code Execution (ACE). Kimi K3 never reached ACE on any of the 41 tasks; the strongest models achieved it on an average of 20 tasks. This means the model can participate in exploit development but has not yet reliably crossed the highest-severity step of a full exploitation chain. [1]

Another test, The Last Ones (TLO), is more like a long-range offensive exercise: 4 subnets, roughly 20 hosts, and a 32-step attack path that takes human experts about 20 hours. Within a 100 million token budget, Kimi K3 progressed to step 17 on average; GLM-5.2 reached step 11; the strongest models averaged step 28.5. Across ten runs, Kimi completed the full path once, while leading models completed it six to seven times. [1]

The subtle conclusion from this data: Kimi K3 already knows how to attack, but it is not yet reliable. Against weak defenses, a pre-planted vulnerability path, and an initial network foothold in a small enterprise network, it is capable of a full end-to-end breach once. Real networks have active defense, alerting, honeypots, and uncertain topology — none of which TLO models — and the test does not penalize actions that would immediately trip alarms in a real SOC. Writing "1/10" up as "can autonomously breach enterprises" is an overstatement; dismissing it as "no threat at all" is dishonest.

Three caveats about testing conditions must also be laid out:

1. US closed-source models were tested with system-level safety guardrails disabled, measuring capability ceilings; public versions typically run with guardrails on. 2. Due to hosting constraints, Kimi K3 only ran a selective subset of tests; its overall cybersecurity capability is estimated mainly from these 41 ExploitBench tasks, so its confidence intervals are wider than for other models. 3. The report explicitly states that Kimi's safeguards did not stop it from assisting with exploit development and offensive operations. That is a risk signal before open-weight release — and it also helps defenders assess in advance.

Secondary coverage goes further, proposing that Kimi's strong general coding but weak cyber offense may relate to "distillation from other frontier models' outputs"; public API safety filtering naturally makes advanced attack samples scarce. This explanation sounds plausible, but it remains speculation. The official NIST/CAISI report does not list distillation as a causal finding, and Kimi's training data has not been made public. At most, the scores show that "general coding leaderboards are not a substitute for cybersecurity capability testing" — they cannot be reverse-engineered into claims about the training recipe. [1][2]

That is the most valuable takeaway today. AI coding is not a single capability axis. Writing business code, finding source-level bugs, reproducing crashes, building stable exploits, and executing long multi-subnet attack chains are related but far from the same problem. Teams that select models based only on SWE-style leaderboards will overestimate the security boundary; teams that look only at Kimi's 0/41 ACE rate will underestimate that it can occasionally complete a full TLO chain.

Kimi K3's 32% is a capability floor and a health check with wide uncertainty. Don't use it as a win/loss poster. Use it for threat modeling — that's where it's worth something.

Sources

1. NIST / CAISI: UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities 2. The Decoder: Kimi K3 cyber evaluation analysis 3. ExploitBench paper