TL;DR
This paper introduces (Net)² , a new benchmark dataset for network traffic analysis, and a novel Multi-Task Hierarchical Learning (MTHL) model. The MTHL model can simultaneously classify network traffic for both malware detection and application identification at different granularities, achieving comparable performance to individual models while significantly reducing training time.
What Problem Does This Paper Address?
The field of network traffic analysis (NTA) is crucial for many applications, from quality of service to security. While machine learning (ML) has shown promise in NTA, a major hurdle is the lack of standardized, comprehensive, and reproducible datasets. Existing datasets are often outdated, specialized for single tasks, or difficult to access, hindering progress and making it hard to compare different approaches. This lack of a common ground makes it challenging to develop generalized ML models for NTA. Furthermore, traditional methods like payload inspection are becoming less effective due to encryption, and port-based approaches are obsolete with dynamic port allocation.
Methods
The core innovation lies in two areas: a new dataset and a new model architecture.
Dataset Creation: The authors curated three datasets:
- (Net)²: A new dataset created from raw traffic captures from Stratosphere IPS, focusing on malware detection.
- CICIDS2017: Re-processed from existing data for malware detection.
- VPN-nonVPN2016: Re-processed from existing data for application classification.
These datasets collectively contain nearly 1.3 million labeled flows and provide features at three granularities: "top-level" (e.g., benign/malware), "mid-level" (e.g., specific malware types or application categories like chat/email), and "fine-grained" (e.g., specific application sub-types like facebook_audio). They extracted a comprehensive set of flow features, including metadata (packet counts, flow duration) and protocol-specific features (TLS, DNS, HTTP).
Multi-Task Hierarchical Learning (MTHL) Model: The proposed MTHL model is designed to handle multiple NTA tasks simultaneously. It features:
- Input Block: Processes raw flow features, including batch normalization and 1D convolutional layers to capture spatial correlations in histogram-like features.
- Residual Blocks: Enable training of deeper networks and improve performance through residual connections.
- Output Block: This is where the hierarchical classification happens. It has separate branches for:
- Mid-level Classification: Uses a fully connected layer with softmax for application and malware type classification.
- Top-level Classification: Includes an additional fully connected layer to incorporate information from the mid-level classification, ensuring hierarchical learning.
The model is trained to perform all tasks at once, leveraging the relationships between different levels of classification.
Datasets
The paper introduces and utilizes three datasets:
- (Net)²: A newly created dataset for malware detection, containing approximately 500k flows from Stratosphere IPS.
- CICIDS2017: A dataset for malware detection, re-processed to yield around 550k flows.
- VPN-nonVPN2016: A dataset for application classification, containing about 163k flows.
These datasets are made publicly available with flow features and annotations to serve as a common benchmark for the research community.
Key Insights and Ideas
- The Need for Unified Benchmarks: The paper strongly emphasizes the critical gap in NTA research due to the lack of standardized, reproducible datasets, hindering direct comparison of methods.
- Hierarchical Granularity is Key: Network traffic can be classified at various levels of detail (e.g., general category vs. specific application). A model that can handle these different granularities simultaneously is more powerful.
- Multi-Task Learning for Efficiency: Training separate models for each NTA task is computationally expensive and time-consuming. A single multi-task model can achieve comparable or better performance with significantly reduced training overhead.
- Feature Engineering Matters: The authors extract a rich set of features, including metadata and protocol-specific ones, highlighting their importance for effective NTA.
- No Single Best Baseline: The evaluation shows that different baseline models excel at different tasks, underscoring the challenge of finding a one-size-fits-all solution and the value of a unified multi-task approach.
Key Results
- The MTHL model achieved comparable or superior F1 scores to the best-performing baseline models across six different scenarios (combinations of dataset and classification level).
- For top-level classifications (e.g., general malware detection), MTHL often outperformed baselines.
- While MTHL's performance on mid-level classifications was slightly lower than the best individual baselines in some cases, it still provided competitive results.
- A significant reduction in training time was observed for the MTHL model compared to training multiple individual baseline models. For instance, training 24 different baseline models could take up to three days, whereas the MTHL model could be trained in a few hours on a GPU.
Limitations and Future Work
Limitations: * The MTHL model's performance on mid-level classifications was slightly less accurate than the best individual baseline models in some scenarios. * The datasets exhibit class imbalance, which the authors acknowledge as a challenge that may bias model predictions. They chose to leave the datasets as-is to present a challenge for researchers.
Future Work: * The authors suggest incorporating raw bytes of traffic data, along with TLS, DNS, and HTTP flow features, to achieve even higher accuracy. * They aim to continue growing the (Net)² datasets to further promote reproducible research. * The work inspires further exploration into how to best leverage hierarchical labeling for more sophisticated NTA.
Summary updated