Multi-Task Hierarchical Learning Based Network Traffic Analytics

Onur Barut, Yan Luo, Tong Zhang, Li Weigang, Peilong Li

TL;DR

This paper introduces (Net)² , a new benchmark dataset for network traffic analysis, and a novel Multi-Task Hierarchical Learning (MTHL) model. The MTHL model can simultaneously classify network traffic for both malware detection and application identification at different granularities, achieving comparable performance to individual models while significantly reducing training time.

What Problem Does This Paper Address?

The field of network traffic analysis (NTA) is crucial for many applications, from quality of service to security. While machine learning (ML) has shown promise in NTA, a major hurdle is the lack of standardized, comprehensive, and reproducible datasets. Existing datasets are often outdated, specialized for single tasks, or difficult to access, hindering progress and making it hard to compare different approaches. This lack of a common ground makes it challenging to develop generalized ML models for NTA. Furthermore, traditional methods like payload inspection are becoming less effective due to encryption, and port-based approaches are obsolete with dynamic port allocation.

Methods

The core innovation lies in two areas: a new dataset and a new model architecture.

Dataset Creation: The authors curated three datasets:

  • (Net)²: A new dataset created from raw traffic captures from Stratosphere IPS, focusing on malware detection.
  • CICIDS2017: Re-processed from existing data for malware detection.
  • VPN-nonVPN2016: Re-processed from existing data for application classification.

These datasets collectively contain nearly 1.3 million labeled flows and provide features at three granularities: "top-level" (e.g., benign/malware), "mid-level" (e.g., specific malware types or application categories like chat/email), and "fine-grained" (e.g., specific application sub-types like facebook_audio). They extracted a comprehensive set of flow features, including metadata (packet counts, flow duration) and protocol-specific features (TLS, DNS, HTTP).

Multi-Task Hierarchical Learning (MTHL) Model: The proposed MTHL model is designed to handle multiple NTA tasks simultaneously. It features:

  • Input Block: Processes raw flow features, including batch normalization and 1D convolutional layers to capture spatial correlations in histogram-like features.
  • Residual Blocks: Enable training of deeper networks and improve performance through residual connections.
  • Output Block: This is where the hierarchical classification happens. It has separate branches for:
  • Mid-level Classification: Uses a fully connected layer with softmax for application and malware type classification.
  • Top-level Classification: Includes an additional fully connected layer to incorporate information from the mid-level classification, ensuring hierarchical learning.

The model is trained to perform all tasks at once, leveraging the relationships between different levels of classification.

Datasets

The paper introduces and utilizes three datasets:

  • (Net)²: A newly created dataset for malware detection, containing approximately 500k flows from Stratosphere IPS.
  • CICIDS2017: A dataset for malware detection, re-processed to yield around 550k flows.
  • VPN-nonVPN2016: A dataset for application classification, containing about 163k flows.

These datasets are made publicly available with flow features and annotations to serve as a common benchmark for the research community.

Key Insights and Ideas

  • The Need for Unified Benchmarks: The paper strongly emphasizes the critical gap in NTA research due to the lack of standardized, reproducible datasets, hindering direct comparison of methods.
  • Hierarchical Granularity is Key: Network traffic can be classified at various levels of detail (e.g., general category vs. specific application). A model that can handle these different granularities simultaneously is more powerful.
  • Multi-Task Learning for Efficiency: Training separate models for each NTA task is computationally expensive and time-consuming. A single multi-task model can achieve comparable or better performance with significantly reduced training overhead.
  • Feature Engineering Matters: The authors extract a rich set of features, including metadata and protocol-specific ones, highlighting their importance for effective NTA.
  • No Single Best Baseline: The evaluation shows that different baseline models excel at different tasks, underscoring the challenge of finding a one-size-fits-all solution and the value of a unified multi-task approach.

Key Results

  • The MTHL model achieved comparable or superior F1 scores to the best-performing baseline models across six different scenarios (combinations of dataset and classification level).
  • For top-level classifications (e.g., general malware detection), MTHL often outperformed baselines.
  • While MTHL's performance on mid-level classifications was slightly lower than the best individual baselines in some cases, it still provided competitive results.
  • A significant reduction in training time was observed for the MTHL model compared to training multiple individual baseline models. For instance, training 24 different baseline models could take up to three days, whereas the MTHL model could be trained in a few hours on a GPU.

Limitations and Future Work

Limitations: * The MTHL model's performance on mid-level classifications was slightly less accurate than the best individual baseline models in some scenarios. * The datasets exhibit class imbalance, which the authors acknowledge as a challenge that may bias model predictions. They chose to leave the datasets as-is to present a challenge for researchers.

Future Work: * The authors suggest incorporating raw bytes of traffic data, along with TLS, DNS, and HTTP flow features, to achieve even higher accuracy. * They aim to continue growing the (Net)² datasets to further promote reproducible research. * The work inspires further exploration into how to best leverage hierarchical labeling for more sophisticated NTA.

Summary updated

Abstract

Classifying network traffic is the basis for important network applications. Prior research in this area has faced challenges on the availability of representative datasets, and many of the results cannot be readily reproduced. Such a problem is exacerbated by emerging data-driven machine learning based approaches. To address this issue, we present(N et)2databasewith three open datasets containing nearly 1.3M labeled flows in total, with a comprehensive list of flow features, for there search community1. We focus on broad aspects in network traffic analysis, including both malware detection and application classification. As we continue to grow them, we expect the datasets to serve as a common ground for AI driven, reproducible research on network flow analytics. We release the datasets publicly and also introduce a Multi-Task Hierarchical Learning (MTHL)model to perform all tasks in a single model. Our results show that MTHL is capable of accurately performing multiple tasks with hierarchical labeling with a dramatic reduction in training time.
BibTeX
@misc{barut2021multi,
  author = {Onur Barut and Yan Luo and Tong Zhang and Li Weigang and Peilong Li},
  title = {{Multi-Task Hierarchical Learning Based Network Traffic Analytics}},
  year = {2021},
  eprint = {2106.03850},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2106.03850},
}
Figure