Abstract
Graph neural network-based network intrusion detection systems have recently demonstrated state-of-the-art performance on benchmark datasets. Nevertheless, these methods suffer from a reliance on target encoding for data pre-processing, limiting widespread adoption due to the associated need for annotated labels—a cost-prohibitive requirement. In this work, we propose a solution involving in-context pre-training and the utilization of dense representations for categorical features to jointly overcome the label-dependency limitation. Our approach exhibits remarkable data efficiency, achieving over 98% of the performance of the supervised state-of-the-art with less than 4% labeled data on the NF-UQ-NIDS-V2 dataset.
Introduction
Modern Network Intrusion Detection Systems (NIDS) are increasingly leveraging Machine Learning, specifically Graph Neural Networks (GNNs), to capture the inherently relational nature of network traffic. Unlike traditional point-based methods, GNNs can identify the structural topology of an attack—such as the coordinated patterns of a Distributed Denial of Service (DDoS) or the sequential steps of a Man-in-the-Middle (MitM) exploit—rather than analyzing isolated packets in a vacuum.
Despite their potential, a significant bottleneck persists: label dependency. Most high-performing GNN-based NIDS rely on "target encoding" for categorical features like protocol types or flags. Target encoding requires access to ground-truth labels (benign vs. malicious) during the data pre-processing stage to calculate statistical correlations. In real-world environments, labeling network traffic is an expensive, labor-intensive process. This creates a circular dependency where a labeled dataset from a specific network is required before a detector can even be built for that environment. This work introduces a framework to break this cycle through self-supervised representation learning.
Methodology
To overcome the limitations of label-dependent pre-processing, we propose two fundamental shifts in the NIDS pipeline: the adoption of dense vector representations and the implementation of in-context pre-training.
Dense Vector Representations
Instead of using scalar-valued target encodings that compress categorical data into a single label-dependent number, we utilize dense embeddings. Categorical features are treated as high-dimensional vectors via one-hot encoding followed by a linear projection. This approach preserves the rich semantic information of network protocols and flags without requiring labels during the initial data preparation phase.
In-Context Pre-training
We introduce a two-stage training pipeline designed to maximize data efficiency:
- Stage 1: Self-Supervised Learning (SSL): We employ an encoder (E-GraphSAGE) and a discriminator in a configuration known as Anomal-E (derived from Deep Graph Infomax). The model analyzes vast quantities of unlabeled traffic from the target network to distinguish between real network structures and "corrupted" structures where features have been randomly shuffled. This "in-context" pre-training allows the model to learn the baseline behavior and structural nuances of a specific network environment.
- Stage 2: Supervised Fine-Tuning: The pre-trained encoder is connected to a classification head and fine-tuned using a very small fraction of labeled data. By starting with a pre-trained state, the model requires significantly fewer examples to reach high accuracy.
Experimental Setup and Datasets
The proposed approach was evaluated on two prominent heterogeneous NIDS benchmarks:
- ToN-IoT: A dataset featuring 40 features across various protocols (TCP, UDP, HTTP, DNS) with nine distinct attack types. It is characterized by a high proportion of categorical features (31 out of 40), making it an ideal test for embedding strategies.
- NF-UQ-NIDS-V2: A massive, standardized NetFlow dataset compiled from four different sources, containing over 75 million records and 20 attack categories. This represents a complex, large-scale simulation of real-world network diversity.
Key Insights
SSL as a Weight Initializer
Self-supervised learning serves as a superior initialization strategy for supervised tasks. By learning the underlying structure of the network first, the GNN begins the supervised phase with an inherent understanding of "normal" traffic patterns, allowing it to generalize from fewer labeled examples.
Dense vs. Scalar Representations
While target encoding is a common shortcut in NIDS research, it results in significant information loss. Dense embeddings capture the nuances of protocol behavior essential for detecting "zero-day" or novel attacks that have not yet been correlated with labels in a training set.
The Importance of "In-Context" Learning
Unlike domains like computer vision, where models are often pre-trained on generic datasets (e.g., ImageNet), NIDS benefits most from pre-training on the specific network it is intended to monitor. This in-context approach ensures the model is tuned to the unique quirks and traffic distributions of the local environment.
Results
The experimental results demonstrate a substantial leap in data efficiency and model stability:
- Extreme Few-Shot Performance: On the NF-UQ-NIDS-V2 dataset, the pre-trained model (D-SSL) achieved an F1-score of 84.64% using only 3.7% of the labeled training data. This represents 95.11% of the performance achieved when using the entire labeled dataset.
- Increased Stability: Pre-training significantly reduced performance variance. On the ToN-IoT dataset, variance decreased by over 82%, indicating that the detector is much more reliable across different training iterations.
- Baseline Comparison: The dense SSL approach consistently outperformed standard GNNs and traditional machine learning models, such as Decision Trees, particularly as dataset complexity increased.
Conclusion and Future Work
This research demonstrates that in-context pre-training and dense representations can effectively mitigate the label-dependency bottleneck in NIDS. However, several avenues for future research remain. Current models utilize a basic GNN backbone (E-GraphSAGE); exploring Graph Transformers may yield further accuracy gains. Additionally, incorporating edge directionality (source vs. destination) is critical for capturing the full context of network flows. Finally, the transition from synthetic benchmarks to live, production-grade network traffic will be the ultimate test of the "Always be Pre-Training" workflow.
Summary updated