Evaluating Open-Weight LLMs for Generating Structured Threat Information (STIX) for Autonomous Vehicle Vulnerabilities

Paper Overview

• Research areas: cs.CR, cs.AI
• Authors: Md Erfan, Ahmed Ryan, Md Kamal Hossain Chowdhury
• arXiv: 2507.15483

Background

Connected and Autonomous Vehicles (CAVs) rely on interconnected software and hardware components — sensors, Electronic Control Units, in-vehicle infotainment systems, and telematics units — where vulnerabilities can compromise assets, users, and vehicle operations. These vulnerabilities are documented as plain text in the Common Vulnerabilities and Exposures (CVE) database, but security practitioners need structured information about affected assets, weakness types, and attack behaviors to mitigate risks effectively.

Contribution

The authors evaluate open-weight Large Language Models (LLMs) for generating Structured Threat Information Expression (STIX) for CAV-related CVEs. They construct CAV-STIXGen, a dataset mapping CAV vulnerability descriptions to:
• STIX Domain Objects (SDO)
• STIX Relationship Objects (SRO)
• Common Weakness Enumeration (CWE)
• MITRE ATT&CK technique mappings

Evaluation

Using this dataset, 11 open-weight LLMs (4B to 120B parameters) were evaluated across various prompting strategies and temperatures.

Single-model results

| Task | F1 Score | |---|---| | SDO | 0.94 | | SRO | 0.63 | | CWE mapping | 0.99 | | MITRE ATT&CK mapping | remains challenging |

Multi-agent setup

• Gemma-4-31B: 0.91 F1 on SDOs
• Codestral-22B: 0.43 F1 on SROs

Threat Pattern Analysis

The authors analyze CWE and MITRE ATT&CK co-occurrences to identify recurring threat patterns in the CAV domain, demonstrating how AI-assisted vulnerability-to-STIX translation can automate threat intelligence and prioritize defense in transportation security.

--- *Auto-collected on 2026-07-21*