Abstract
Cyber-attack attribution is a critical but labor-intensive task in threat intelligence. This paper investigates the use of Large Language Models (LLMs) and vector embedding (VE) models to automate the extraction of Tactics, Techniques, and Procedures (TTPs) from unstructured forensic reports. By building a probabilistic attribution pipeline, the researchers demonstrate that even "noisy" AI-generated TTP sets can effectively characterize threat actor behavior, significantly outperforming random baselines and narrowing the search space for human analysts.
Introduction
Attributing a cyber-attack to a specific threat actor, such as a nation-state group, remains a significant bottleneck in cybersecurity operations. Analysts typically manually review dense forensic documentation to identify TTPs—behavioral indicators that are much harder for attackers to spoof than low-level indicators like IP addresses. This manual process leads to delays in defensive and diplomatic responses. This study explores whether off-the-shelf AI tools can bridge this gap by transforming raw threat intelligence into structured profiles for automated attribution.
Methodology
The researchers developed a two-step pipeline. First, TTPs are identified using two distinct methods: direct LLM prompting (GPT-4) and Vector Embedding (VE) search using OpenAI’s text-embedding-3-large model to match document chunks to MITRE ATT&CK definitions. Second, a probabilistic model based on Bayes’ Theorem calculates the likelihood of a specific threat actor given the extracted TTPs. The model utilizes a weights matrix representing $P(\text{TTP} | \text{Actor})$ and incorporates "Expert Priors" to account for the historical frequency of specific actors.
The Noise Paradox and Behavioral Signals
A key finding of the study is the "Noise Paradox." While GPT-4 and VE search methods produced TTP sets with relatively low overlap (Jaccard similarity) compared to human-curated datasets, the overall frequency of TTPs identified by GPT-4 showed a strong correlation ($R^2 = 0.81$) with human data. This indicates that while AI may identify more TTPs than a human analyst (introducing "noise"), it accurately captures the underlying behavioral "signal" of the threat actor.
Furthermore, the VE search approach provides an audit trail by mapping specific text chunks to TTPs, offering greater transparency than raw LLM responses. Hallucination rates remained remarkably low (under 1%), with most discrepancies involving deprecated MITRE IDs.
Attribution Performance
The pipeline's performance was measured by the average rank of the correct threat actor among 29 candidates. While a random baseline yields an average rank of 15, the automated pipeline significantly improved these results. The most effective configuration, combining Hypothetical Document Embeddings (HyDE) with Expert Priors, achieved an average rank of 7.55.
Conclusion
The study demonstrates that automated TTP extraction and attribution are viable using current LLM and embedding technologies. While challenges remain—such as the high entropy of certain actors like the Lazarus Group and the need to balance specificity with noise—the proposed pipeline serves as a powerful tool to augment human analysis. Future work focusing on fine-tuning models on annotated TTP data and integrating real-time intelligence feeds could further enhance attribution accuracy.
Summary updated