Evaluating Open-Weight LLMs for Generating Structured Threat Information (STIX) for Autonomous Vehicle Vulnerabilities

Forum topic · 小凯 · 2026-07-21

Summary

Connected and Autonomous Vehicles (CAVs) depend on interconnected sensors, electronic control units, infotainment systems, and telematics units, whose vulnerabilities are documented as plain text in CVE databases. Security practitioners need structured details on affected assets, weaknesses, and attack behaviors to mitigate risks. This paper evaluates open-weight large language models for converting CAV-related CVEs into STIX, a structured threat intelligence format. The authors build CAV-STIXGen, a dataset mapping CAV vulnerability descriptions to STIX domain objects (SDO), STIX relationship objects (SRO), CWE identifiers, and MITRE ATT&CK techniques. They test 11 open-weight LLMs (4B–120B parameters) across prompting strategies and temperature settings. Single-model configurations reach F1 scores of 0.94 for SDO generation, 0.63 for SRO generation, and 0.99 for CWE mapping, while full MITRE ATT&CK mapping remains difficult. In multi-agent setups, Gemma-4-31B and Codestral-22B achieve F1 scores of 0.91 (SDO) and 0.43 (SRO). The paper also analyzes CWE and ATT&CK co-occurrences to identify recurring threat patterns in the CAV domain, showing how AI-assisted vulnerability-to-STIX translation can automate threat intelligence and prioritize transportation security defenses. arXiv: 2507.15483.

Paper Overview

Research areas: cs.CR, cs.AI Authors: Md Erfan, Ahmed Ryan, Md Kamal Hossain Chowdhury Published: 2026-07-21 arXiv: 2507.15483

Abstract

Connected and Autonomous Vehicles (CAVs) rely on interconnected software and hardware components, including sensors, Electronic Control Units, in-vehicle infotainment systems, and telematics units, where vulnerabilities can compromise assets, users, and vehicle operations. These vulnerabilities are commonly documented as plain text in the Common Vulnerabilities and Exposures (CVE) database; however, security practitioners require structured information about affected assets, types of weaknesses, and attack behaviors to effectively mitigate the risks from these vulnerabilities.

To this end, the authors evaluate open-weight Large Language Models (LLMs) for generating Structured Threat Information Expression (STIX), a well-known structured format for representing threat information, for CAV-related CVEs.

Methodology

A dataset called CAV-STIXGen maps CAV vulnerability descriptions to:
STIX domain objects (SDO)
STIX relationship objects (SRO)
Common Weakness Enumeration (CWE)
MITRE ATT&CK technique mappings
11 open-weight LLMs (4B to 120B parameters) were evaluated across various prompting strategies and temperature settings.

Key Results

Single-model configurations achieve F1 scores of:
0.94 for SDO generation
0.63 for SRO generation
0.99 for CWE mapping
Complete MITRE ATT&CK mapping remains challenging.
In a multi-agent setup, Gemma-4-31B and Codestral-22B achieve F1 scores of 0.91 for SDOs and 0.43 for SROs, respectively.
Analysis of CWE and MITRE ATT&CK co-occurrences identifies recurring threat patterns in the CAV domain.

Conclusion

The study demonstrates how AI-assisted vulnerability-to-STIX translation can automate threat intelligence and help prioritize defenses in transportation security.