IoT-Portrait: Automatically Identifying IoT Devices via Transformer with Incremental Learning

Juan Wang, Zhong Jing, Jiangqi LiFuture 2023 · DOI

TL;DR

This paper introduces IoT-Portrait, an automated system for identifying Internet of Things (IoT) devices by analyzing their network traffic using a transformer model. It uniquely combines active device scanning for labeling with class incremental learning to continuously adapt to new devices without forgetting previously learned ones, achieving high accuracy and robustness.

What Problem Does This Paper Address?

The proliferation of IoT devices in smart homes and cities presents a significant challenge for network management and security. Identifying these devices is crucial for understanding network composition, detecting anomalies, and defending against cyber threats. However, existing methods often rely on extensive manual labeling of traffic data, which is time-consuming and impractical as the number and diversity of IoT devices grow. Furthermore, when new devices are introduced, traditional machine learning models struggle with "catastrophic forgetting," where learning new device types leads to a drastic loss of performance on previously known devices. This paper aims to solve these issues by proposing an automated, continuously learning system for IoT device identification.

Methods

IoT-Portrait employs a two-pronged approach:

  • Automated Labeling via Active Scanning: Instead of manual labeling, IoT-Portrait uses a combination of active scanning tools (Nmap, ARE, cURL) to gather device information like OS, manufacturer, and product model. This information is then used as labels for the network traffic data. This significantly reduces the manual effort required for training.
  • Transformer Network for Traffic Analysis: The core of IoT-Portrait's identification engine is a transformer neural network. This architecture is chosen for its ability to effectively model sequential data like network traffic, capturing temporal dependencies and relationships between packets without requiring manual feature engineering.
  • Class Incremental Learning (FTDL): To address catastrophic forgetting, IoT-Portrait integrates a class incremental learning method called Fine Tuning with Distillation Loss (FTDL). This approach allows the model to learn new device types while retaining knowledge of existing ones. It achieves this by using distillation loss to guide the new model's predictions to mimic the old model's outputs, preserving past knowledge with minimal storage overhead for old exemplars.

Datasets

The paper utilizes two primary sources for evaluation:

  • Lab Environment: A custom setup with six IoT devices (three smart cameras, two central controllers, and a smart light) was used for initial testing and demonstration.
  • IoT Trace Dataset: A publicly available dataset from UNSW containing network traffic from 21 different IoT devices. For their experiments, the authors selected 15 devices from this dataset due to variations in traffic volume.

Key Insights and Ideas

  • Traffic as a Unique Fingerprint: The research confirms that IoT devices exhibit distinct and regular patterns in their network traffic (e.g., specific ports, communication endpoints, packet rates) that can serve as reliable "fingerprints" for identification.
  • Automated Labeling is Feasible and Efficient: Moving away from manual labeling by integrating active scanning significantly streamlines the device identification pipeline, making it scalable.
  • Transformer's Power for Sequential Data: The transformer architecture proves highly effective at learning complex patterns from time-series network traffic data, outperforming traditional methods that might require more manual feature engineering.
  • Class Incremental Learning is Crucial for Evolving IoT Environments: The successful application of FTDL demonstrates that continuous learning without catastrophic forgetting is achievable and essential for systems that need to adapt to new devices over time.

Key Results

  • High Identification Accuracy: IoT-Portrait achieved an impressive identification rate of up to 99% in their lab environment and over 96% on the IoT Trace dataset, even for devices of the same type from different manufacturers.
  • Robustness to Catastrophic Forgetting: The FTDL method demonstrated strong resistance to catastrophic forgetting, maintaining an accuracy of over 85% after seven incremental states, with a negligible increase in memory and time costs compared to retraining from scratch.
  • Efficient Resource Usage: The FTDL method requires only 1.8 MB of memory to store exemplars for incremental learning, a significant reduction compared to storing all training samples (96.1 MB for the full model). Prediction time was also comparable to the full model.

Limitations and Future Work

  • Network Stability Dependency: The model's performance is sensitive to network stability. Fluctuations or errors in network traffic could impact identification accuracy. The authors suggest averaging traffic data over time intervals to mitigate this.
  • Limited Incremental Learning Methods Tested: While FTDL performed well, the paper only compared it against a few other class incremental learning methods. Future work could explore a wider range of techniques.
  • Further Exploration of Feature Engineering: While the transformer reduces the need for manual feature engineering, further research into feature engineering for network traffic could potentially enhance performance.
  • Adapting to Network Fluctuations: Investigating strategies to adapt to natural network diversity and fluctuations is a key area for future research.

Summary updated

Abstract

With the development of IoT, IoT devices have proliferated. With the increasing demands of network management and security evaluation, automatic identification of IoT devices becomes necessary. However, existing works require a lot of manual effort and face the challenge of catastrophic forgetting. In this paper, we propose IoT-Portrait, an automatic IoT device identification framework based on a transformer network. IoT-Portrait automatically acquires information about IoT devices as labels and learns the traffic behavior characteristics of devices through a transformer neural network. Furthermore, for privacy protection and overhead reasons, it is not easy to save all past samples to retrain the classification model when new devices join the network. Therefore, we use a class incremental learning method to train the new model to preserve old classes’ features while learning new devices’ features. We implement a prototype of IoT-Portrait based on our lab environment and open-source database. Experimental results show that IoT-Portrait achieves a high identification rate of up to 99% and is well resistant to catastrophic forgetting with a negligible added cost both in memory and time. It indicates that IoT-Portrait can classify IoT devices effectively and continuously.
BibTeX
@inproceedings{wang2023iot,
  author = {Juan Wang and Zhong Jing and Jiangqi Li},
  title = {{IoT-Portrait: Automatically Identifying IoT Devices via Transformer with Incremental Learning}},
  booktitle = {Future},
  year = {2023},
  doi = {10.3390/fi15030102},
}
Figure