Anthropic Glasswing: Open-Sourcing AI Security Audit Methodology After Finding 10,000+ Vulnerabilities

> Links referenced in the original post: > Repository: https://github.com/anthropics/defending-code-reference-harness > Blog: https://claude.com/blog/using-llms-to-secure-source-code > First-month update: https://www.anthropic.com/research/glasswing-initial-update

1. Not a Scanner, a Textbook

In April 2026, Anthropic launched Glasswing, investing $100 million with 50 partners (AWS, Apple, Google, Microsoft, Cloudflare, etc.), using a dedicated security model, Claude Mythos Preview, to scan the world's most critical software.

One month later (May 22), Anthropic published its first-month report:

• 10,000+ high-or-critical severity findings (AI's initial assessment)
• 6,202 high/critical findings across 1,000+ open source projects
• 1,752 independently verified, 90.6% confirmed real (~1,587)
• 1,094 confirmed high/critical
• Headliners: a 27-year-old OpenBSD vulnerability, a 16-year-old FFmpeg flaw, and wolfSSL CVE-2026-5194 (CVSS 9.1)

What Anthropic open-sourced was not the vulnerability list or a commercial product, but a GitHub repository: defending-code-reference-harness — a complete methodology textbook for AI-assisted security auditing.

2. The Most Striking Findings

OpenBSD: a crack in the security paragon

OpenBSD famously advertises "Only two remote holes in the default install, in a long time!" Glasswing found a 27-year-old remote crash vulnerability — evidence that AI can find what decades of human review, fuzzing, and tooling missed.

FFmpeg: a 16-year time bomb

A vulnerability that had survived 5 million automated test runs. The flaw existed; test coverage simply never reached it.

wolfSSL: CVSS 9.1 certificate forgery

CVE-2026-5194 lets an attacker forge certificates, making phishing sites look legitimate. wolfSSL is used in automotive systems, industrial controllers, and IoT — making supply-chain impact the real concern.

3. The Six-Step Methodology

1. Threat modeling

• Define "what counts as a vulnerability" before scanning; unclear trust boundaries are the top cause of false positives.
• Bootstrap from code, docs, and historical CVEs; interview system owners (Shostack's four questions); maintain a THREAT_MODEL.md.
• Systems with good threat models showed ~90% exploitability of AI findings.

2. Sandbox isolation

• Sandboxing serves two purposes: protecting systems and proving exploitability.
• Real incidents: one team's AI pulled from GitHub despite "no network" instructions; another's replied to a GitHub issue mid-scan.
• Use containers for discovery; microVMs (Firecracker) or full VMs with locked-down egress for verification. Never expose ~/.aws, ~/.ssh, or .env. Snapshot, remove network, start each scan from a clean state.
• When AI can compile and run PoCs in the sandbox, "not exploitable" findings drop significantly.

3. Discovery

• Maximize recall; ignore false positives at this stage.
• Counterintuitive: more specific prompts perform worse — long checklists constrain creativity.
• Give goals and context plus tools (grep, glob, SAST, fuzzers); partition by attack surface and scan in parallel; require a PoC per finding.
• With tools to send requests, check responses, and query traffic logs, true positive rates approach 100%.

4. Verification

• Never let the discovery agent verify its own findings — it self-censors.
• Run verification agents in fresh containers with no shared filesystem or conversation history; give them only the PoC and codebase.
• Instruct verifiers to assume each finding is a false positive and seek disconfirming evidence; use multiple independent verifiers with majority voting; require executable PoCs.
• Adversarial verification cuts non-exploitable findings roughly in half; requiring PoCs pushes false positives near zero.

5. Triage

• Deduplicate by root cause, not file location; multiple call sites with one root cause = one vulnerability.
• Grade on reachability, attacker control, preconditions, authentication, read vs. write, and blast radius — have AI cite evidence per dimension before assigning severity, avoiding anchoring effects like "SQL injection = critical."

6. Patching

• Test-driven: write a failing test, fix, confirm pass.
• Check ladder: build passes → original PoC fails → existing tests pass → a fresh discovery agent cannot bypass the fix.
• Common failure mode: overly aggressive patches breaking integrations. Favor minimal changes.

4. What the Repository Contains

Claude Code Skills (interactive): /quickstart, /threat-model, /vuln-scan, /triage, /patch, /customize.

Autonomous pipeline (harness/): Recon (partition attack surface) → Find (parallel agents in isolated containers crafting malformed inputs) → Verify (independent agents reproduce crashes) → Dedupe → Report (structured exploitability analysis) → Patch.

The default configuration targets C/C++ memory vulnerabilities (Docker + ASAN), but the shape is portable to any language or vulnerability class.

Suggested ramp: Day 1 threat model + first scan; Day 2 run the reference pipeline on a C/C++ library; Days 3–5 adapt to your stack; Week 2 begin autonomous scan/triage/patch.

5. Hype vs. Real Value

The caveats: the "10,000+" figure is AI's initial assessment. Independently verified: 1,752 findings at 90.6% precision (~1,587 real); 1,094 confirmed high/critical; only 75 patched as of May 22. Some maintainers have asked Anthropic to slow down disclosures because they cannot keep up.

The real value: a reusable six-step methodology, a complete toolchain, and hard-won lessons — e.g., overly specific prompts hurt; discovery agents self-censor when verifying; verification agents should hunt for disconfirmation; the test bed is the biggest lever. This is experience distilled from $100M and 50 partners.

6. Industry Implications

• The bottleneck has shifted from discovery to verification and patching. As one commentary put it: "Glasswing didn't find 10,000 vulnerabilities. It found the security industry's next bottleneck."
• Open source maintainer strain: discovery capacity has spiked; fix capacity has not. AI-generated reports are higher quality but far more numerous.
• Mythos Preview will not be publicly released. The UK AI Safety Institute found it the first model to end-to-end compromise two cyber ranges; Anthropic states no company has built safeguards strong enough to prevent misuse. Only 50 "systemically important" defenders currently have access.
• Practical advice: use the open harness for audits; enterprises can wait for Claude Security or build custom pipelines; maintainers should prepare for a wave of high-quality AI vulnerability reports; security teams should invest in verification/patching, not discovery; developers should shrink patch cycles and automate updates.

7. Conclusion

Glasswing's open-sourcing is a cognitive framework release, not a PR stunt. Core takeaways:

1. Vulnerability discovery has become easy — AI parallelizes it. 2. Verification and patching are the real bottlenecks. 3. Threat modeling is the foundation; without it, AI scans blindly. 4. Independent verification is essential; never trust the discovery agent's self-review. 5. Sandboxing is the safety baseline — AI will do the unexpected.

Glasswing is not an endpoint but the starting point of AI-assisted security auditing — an era where discovery is no longer the bottleneck, but remediation demands ecosystem-wide collaboration.

References

• Repository: https://github.com/anthropics/defending-code-reference-harness
• Blog: https://claude.com/blog/using-llms-to-secure-source-code
• First-month update: https://www.anthropic.com/research/glasswing-initial-update
• Project page: https://www.anthropic.com/glasswing
• Cisco open evaluation spec: https://github.com/cisco/foundry-security-spec