Abstract
Indicators of Compromise (IOCs) are artifacts observed on a network or in an operating system that can be utilized to indicate a computer intrusion and detect cyber-attacks in an early stage. Thus, they exert an important role in the field of cybersecurity. However, state-of-the-art IOCs detection systems rely heavily on hand-crafted features with expert knowledge of cybersecurity, and require large-scale manually annotated corpora to train an IOC classifier. In this paper, we propose using an end-to-end neural-based sequence labelling model to identify IOCs automatically from cybersecurity articles without expert knowledge of cybersecurity. By using a multi-head self-attention module and contextual features, we find that the proposed model is capable of gathering contextual information from texts of cybersecurity articles and performs better in the task of IOC identification. Experiments show that the proposed model outperforms other sequence labelling models, achieving the average F1-score of 89.0% on English cybersecurity article test set, and approximately the average F1-score of 81.8% on Chinese test set.
Introduction
In the cybersecurity domain, Indicators of Compromise (IOCs) serve as critical forensic artifacts—such as MD5 hashes of malware, malicious domain names, or command-and-control IP addresses—that allow security analysts to detect and mitigate threats. However, the vast majority of threat intelligence is trapped in unstructured formats like blog posts and technical reports. Manually extracting this data is labor-intensive, and traditional automated systems often rely on rigid, hand-crafted rules that struggle to keep pace with the rapidly evolving threat landscape.
To address these challenges, researchers have developed an end-to-end neural sequence labelling model designed to automatically extract IOCs from unstructured text. By leveraging multi-head self-attention and automatically extracted contextual features, the model achieves high performance without requiring deep domain expertise or manual rule-writing.
Methodology
The proposed architecture is built upon the Bi-LSTM-CRF (Bidirectional Long Short-Term Memory with Conditional Random Fields) framework, enhanced with several sophisticated modules to capture both local and global context.
The model utilizes a multi-layered approach:
- Hybrid Embeddings: Tokens are represented using a combination of word-level embeddings (Word2vec) and character-level embeddings generated via a Bi-LSTM. This hybrid approach is particularly effective for handling "out-of-vocabulary" technical terms common in cybersecurity reports.
- Multi-Head Self-Attention: Borrowing from the Transformer architecture, this module allows the model to weigh the relative importance of different words within a sentence, capturing long-range dependencies that standard LSTMs might overlook.
- Contextual Features: The model automatically identifies "contextual keywords" (e.g., "download," "malware") from the training data. The co-occurrence frequency of these keywords serves as a critical feature, mimicking how human analysts use surrounding text to validate potential IOCs.
- CRF Layer: A final Conditional Random Field layer ensures the predicted label sequences are logically consistent according to the BIO (Begin, Inside, Outside) tagging scheme.
Experimental Setup
The model was evaluated on two distinct datasets:
- English Dataset: Comprising 687 Advanced Persistent Threat (APT) reports published between 2008 and 2018, with 370 manually annotated for training and testing.
- Chinese Dataset: Consisting of 5,427 articles crawled from 35 cybersecurity blogs, with 607 manually annotated.
The task involved identifying 11 distinct labels, including attacker identities, attack methods, file hashes, and CVE vulnerabilities.
Results and Discussion
The proposed model consistently outperformed state-of-the-art baselines across both datasets. On the English test set, the model achieved a high precision of 93.1% and an F1-score of 89.0%.
Key insights from the results include:
- The Power of Context: Incorporating contextual features significantly improved the model's ability to distinguish between actual IOCs and benign strings that merely look like them (e.g., a legitimate IP address vs. a malicious one).
- Attention to Detail: The multi-head attention mechanism proved vital for processing long lists of IOCs, effectively linking tokens to descriptive verbs located elsewhere in the text.
- Bilingual Synergy: Interestingly, training on a combined English and Chinese dataset improved performance on both, with the Chinese F1-score rising to 84.4% in some tests. This suggests that the underlying contextual patterns used to describe cyber threats are largely universal across languages.
Conclusion
This neural-based approach demonstrates that high-accuracy IOC extraction is possible without heavy reliance on hand-crafted rules. While the model shows robust performance, future work could integrate deeper contextual embeddings from models like BERT or explore semi-supervised learning to leverage the vast amounts of unlabelled cybersecurity text available online. Additionally, extending the model to capture more complex, behavioral attack patterns remains a promising direction for future research.
Summary updated