Social-HUMINT: An LLM-Augmented Framework for Measuring Social Engineering Susceptibility

Forum topic · 小凯 · 2026-10-25

Summary

Social-HUMINT (arXiv:2610.31573) by Pellegrino Casoria and Simon Pietro Romano unifies SOCMINT and Cyber-HUMINT into a single doctrine-grounded taxonomy of 38 capabilities spanning the cyber threat intelligence lifecycle, validated by domain experts with a Fleiss' kappa of 0.92. In the post-API landscape, large language models can extract structured information from unstructured online content and deduce attributes that users never explicitly disclosed. The authors operationalize the taxonomy's core path—extraction, inference, and susceptibility assessment—as an LLM-augmented multi-agent pipeline. Extraction achieves an F1 score of 97.3%, while inferred attributes receive an average rating of 4.73/5 versus baselines. Susceptibility is evaluated indirectly: the pipeline predicts how each target would rank personalized engagement options, matching actual choices significantly above chance. The framework augments security analysts' tradecraft and enables defenders to gauge their own exposure before adversaries exploit it.

Paper Overview

Field: Machine Learning Authors: Pellegrino Casoria, Simon Pietro Romano Published: 2026-10-22 arXiv: 2610.31573

Abstract

In the post-API landscape, LLMs can extract structured information from unstructured content and deduce attributes users never disclosed. We unify SOCMINT and Cyber-HUMINT into a single doctrine-grounded taxonomy of 38 capabilities across the CTI lifecycle, validated by domain experts (Fleiss' κ = 0.92). We operationalize its core path as an LLM-augmented multi-agent pipeline. Extraction reaches F1 = 97.3%, and inferred attributes score 4.73/5. Susceptibility is assessed indirectly: the pipeline predicts how each target would rank personalized engagement options, matching their actual choices significantly above chance. The framework augments security analysts' tradecraft and enables defenders to gauge their own exposure before an adversary does.

Key Results

Unified taxonomy: 38 capabilities merging SOCMINT and Cyber-HUMINT, grounded in intelligence doctrine and covering the full CTI lifecycle; expert validation achieved Fleiss' κ = 0.92.
LLM multi-agent pipeline: Implements the core path of extraction, inference, and susceptibility assessment.
Extraction performance: F1 = 97.3% for structured information extraction from unstructured content.
Inference quality: Inferred attributes (never explicitly disclosed by users) rated 4.73/5, well above baselines.
Indirect susceptibility assessment: The pipeline predicts targets' rankings of personalized engagement options, matching actual choices significantly above chance.

Implications

The framework augments security analysts' tradecraft and, importantly, allows defenders to evaluate their own exposure before an adversary does.