AutoML4ETC: Automated Neural Architecture Search for Real-World Encrypted Traffic Classification

Navid Malekghaini, Elham Akbari, Mohammad Ali Salahuddin, Noura Limam, Raouf Boutaba, Bertrand Mathieu, Stephanie Moteau, Stéphane Tuffin

TL;DR

This paper introduces AutoML4ETC, a novel automated tool that designs efficient and high-performing neural network architectures for encrypted traffic classification. It addresses the challenge of performance decay in deployed models due to data drift by automating the architecture search process, leading to models that are both more accurate and significantly lighter than existing state-of-the-art solutions.

What Problem Does This Paper Address?

Traffic Classification (TC) is crucial for network management, security, and quality of service. However, with the widespread adoption of encryption (like HTTPS via TLS), classifying network traffic has become increasingly difficult. While deep learning (DL) has shown promise, models trained on specific datasets often experience performance degradation over time in real-world production environments. This "data drift" necessitates frequent, manual re-tuning of model architectures and hyperparameters, which is time-consuming and requires significant domain expertise. The paper aims to automate this process by developing a tool that can automatically design neural architectures tailored to specific datasets, leading to efficient and accurate encrypted traffic classifiers that adapt to evolving traffic patterns.

Methods

The core of this work is AutoML4ETC, a framework for automated neural architecture search (NAS) specifically for encrypted traffic classification. Key components include:

  • Novel Search Space: A custom search space is designed, focusing on raw packet header bytes. This space is built from building blocks (operations, connections, layer types) carefully selected based on existing literature and empirical analysis of state-of-the-art ETC models. It's structured around "Normal" and "Reduction" cells, aiming for lightweight and generalizable models.
  • Search Algorithms: The framework is modular, allowing integration of various search algorithms. The paper evaluates Reinforcement Learning (RL), Monte Carlo Tree Search (MCTS), Evolutionary Algorithms (EA), and Random Search (RS), finding that RS performs competitively, especially when paired with a well-designed search space.
  • Child Model Training Strategies: Two strategies are explored: "full training" (training child models until convergence) and "partial training" (training for fewer epochs and then fine-tuning the best candidate). The study finds that for their specific search space, full training for around 40 epochs yields better accuracy with fewer parameters.
  • Early Classification Focus: The search space is specifically tailored to enable classification using only the first few packets of a flow, which is critical for real-time applications.

Datasets

The paper utilizes a combination of real-world and public benchmark datasets:

  • Real-world Orange Mobile Network Data: TLS and QUIC traffic collected from the Orange mobile network, captured at different times (e.g., July 2019, September 2020, April 2021, May 2021, June 2021). These datasets are labeled at both service and application levels.
  • Public Benchmark Datasets:
    • ISCXVPN2016 (mixed protocol environment)
    • UCDavis QUIC [37] (synthetic QUIC traffic)

The pre-processing pipeline involves removing packet payloads, extracting TLS/QUIC handshake headers, and then labeling flows based on the TLS Server Name Indication (SNI) field. SNIs are obfuscated, and an approximate labeling function using a lookup table is employed.

Key Insights and Ideas

  • Data Drift is a Major Challenge: The paper strongly emphasizes that performance decay in deployed DL models for encrypted traffic is a significant problem, and simply retraining models on new data isn't always sufficient.
  • Automated Architecture Design is Key: AutoML4ETC demonstrates that automating the search for optimal neural architectures can effectively combat data drift and produce models that are both accurate and efficient.
  • Search Space Design Matters More Than Search Algorithm (for efficiency): The study highlights that a well-designed, targeted search space can lead to competitive results even with simpler search algorithms like Random Search, especially when aiming for efficiency and fewer trials.
  • Lightweight Architectures are Achievable: AutoML4ETC consistently generates models with significantly fewer parameters than existing state-of-the-art models, making them more suitable for production environments.
  • Early Classification is Feasible: The proposed architectures can achieve high accuracy using only the initial packets of a flow, which is a crucial capability for real-time traffic classification.

Key Results

  • Superior Accuracy: AutoML4ETC-generated models outperform state-of-the-art ETC models by 1% to 60.1% across various datasets in service-level classification.
  • Significant Efficiency Gains: The generated models are substantially lighter, with over 50 times fewer parameters on average compared to state-of-the-art models.
  • 100% Accuracy on QUIC-UCDavis: A generated architecture achieved 100% accuracy on the QUIC-UCDavis dataset with significantly fewer parameters than other models.
  • Competitive Search Algorithm Performance: Random Search (RS) proved to be a strong contender, achieving results comparable to more complex algorithms like RL, MCTS, and EA, especially when paired with the AutoML4ETC search space.
  • Optimal Training Strategy: For their search space, a full training strategy of approximately 40 epochs for child models was found to be optimal, balancing accuracy and parameter count.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that while partial training can reduce search time significantly, it might result in a slightly less optimal child model compared to full training.
  • The current search space is tailored for raw packet byte-based classification.
  • Future Work:
    • Investigate the inclusion of Attention Networks and Transformer-based architectures within the NAS search space, as these have shown promise in ETC.
  • Explore more efficient one-shot NAS approaches for searching better performing neural architectures.

Summary updated

Abstract

Deep learning (DL) has been successfully applied to encrypted network traffic classification in experimental settings. However, in production use, it has been shown that a DL classifier's performance inevitably decays over time. Re-training the model on newer datasets has been shown to only partially improve its performance. Manually re-tuning the model architecture to meet the performance expectations on newer datasets is time-consuming and requires domain expertise. We propose AutoML4ETC, a novel tool to automatically design efficient and high-performing neural architectures for encrypted traffic classification. We define a novel, powerful search space tailored specifically for the early classification of encrypted traffic using packet header bytes. We show that with different search strategies over our search space, AutoML4ETC generates neural architectures that outperform the state-of-the-art encrypted traffic classifiers on several datasets, including public benchmark datasets and real-world TLS and QUIC traffic collected from the Orange mobile network. In addition to being more accurate, AutoML4ETC's architectures are significantly more efficient and lighter in terms of the number of parameters. Finally, we make AutoML4ETC publicly available for future research.
BibTeX
@article{malekghaini2023automl,
  author = {Navid Malekghaini and Elham Akbari and Mohammad Ali Salahuddin and Noura Limam and Raouf Boutaba and Bertrand Mathieu and Stephanie Moteau and Stéphane Tuffin},
  title = {{AutoML4ETC: Automated Neural Architecture Search for Real-World Encrypted Traffic Classification}},
  journal = {IEEE Trans. Netw. Serv. Manag.},
  year = {2023},
}
Figure