Kimi K3's Cybersecurity Report Card: 32% on ExploitBench, Zero ACE Across 41 V8 Vulnerabilities

A report card that is easy to misread.

The UK AI Security Institute and the US CAISI jointly tested Kimi K3. The three headline numbers circulating in media coverage: Kimi K3 scored roughly 32.2% on ExploitBench, GLM-5.2 around 24.4%, and the strongest group of US frontier models about 76.2%. The gap is large. But writing it off as "44 points behind" misses the most useful parts of this evaluation.[1][2]

ExploitBench is not an ordinary coding benchmark. It collects 41 Chrome V8 vulnerabilities from after 2023 and asks the model to climb a real exploitation ladder: covering and reproducing the crash, arbitrary read/write, control-flow hijack, up to Arbitrary Code Execution (ACE). Kimi K3 never reached ACE on any of the 41 tasks; the strongest models did so on an average of 20 tasks. This means it can participate in exploit development but has not yet reliably crossed the highest-severity, full exploit chain.[1]

Another test, The Last Ones (TLO), is more like a long-horizon attack exercise: 4 subnets, roughly 20 hosts, a 32-step attack path — human experts need about 20 hours. Kimi K3 progressed to an average of step 17 within a 100 million token cap; GLM-5.2 reached step 11; the strongest models averaged 28.5 steps. Across ten runs, Kimi completed the full chain once, while leading models completed it six to seven times.

The data support a nuanced conclusion: Kimi K3 already knows how to attack, but it is not reliable yet. Against weak defenses, pre-planted vulnerabilities, and a small enterprise network with a given initial foothold, it has demonstrated the ability to run the full chain once. Real networks have active defense, alerting, honeypots, and uncertain topology — none of which TLO models — and the test does not penalize actions that would immediately trigger alarms in a real SOC. Calling 1/10 "able to autonomously breach enterprises" is overblown; calling it "no threat at all" is dishonest.

Three methodological caveats must also be laid out:

1. The US closed-source models were tested with system-level safety guardrails disabled, measuring capability ceilings; public versions typically run with guardrails enabled. 2. Due to hosting constraints, Kimi K3 only ran a selective subset of tests; its overall cyber capability was mainly estimated from the 41 ExploitBench tasks, with wider confidence intervals than the other models. 3. The report explicitly states that Kimi's safeguards did not prevent it from assisting with exploit development and offensive operations — a pre-release risk signal for open weights, and useful for defenders' early assessment.

Secondary coverage further suggests that Kimi's strong general coding but weak offensive cyber performance may relate to "distillation from other frontier models' outputs," and that public API safety filtering naturally makes advanced attack samples scarce. Plausible — but still speculation. The official NIST/CAISI report does not list distillation as a causal finding, and Kimi's training data is not public. The existing scores at most show that "general coding leaderboards cannot substitute for cybersecurity capability tests," not that any training recipe can be inferred.[1][2]

That is today's key takeaway. AI coding ability is not a single capability axis. Writing business code, finding source-level bugs, reproducing crashes, building stable exploits, and completing long multi-subnet attack chains are related, but far from the same problem. Teams choosing models based only on SWE-style leaderboards will overestimate security boundaries; those looking only at Kimi's 0/41 ACE will underestimate that it can occasionally complete a full chain in TLO.

Kimi K3's 32% is a capability floor and a health check with wide uncertainty. Don't use it as a win/lose poster. Use it for threat modeling — that's where it's valuable.

Sources

1. NIST / CAISI: UK AISI / CAISI Preliminary Assessment of Kimi K3's Cyber Capabilities 2. The Decoder: Kimi K3 cyber evaluation analysis 3. ExploitBench paper