Cost-aware Feature Selection for IoT Device Classification

Biswadeep Chakraborty, Dinil Mon Divakaran, Ido Nevat, Gareth W. Peters, Mohan Gurusamy

Abstract

IoT device classification is a critical component of network security, yet traditional machine learning approaches often overlook the resource costs associated with feature extraction. This paper introduces a cost-aware framework that treats feature selection as a constrained optimization problem. By incorporating a Loss Matrix to quantify the severity of misclassifications and employing a Cross-Entropy-based optimization algorithm, the researchers demonstrate a method to maintain high classification accuracy within strict computational, memory, and privacy budgets.

Introduction

In modern network environments, identifying the specific type and brand of IoT devices is the first step toward implementing effective security measures, such as targeted patching or firewall rules. However, the assumption that all network traffic features—ranging from simple packet counts to complex deep packet inspection—are available for free is unrealistic. In practice, feature extraction imposes significant costs:

  • Computational Cost: Complex signature matching is CPU-intensive compared to simple packet counting.
  • Memory Cost: Tracking state for numerous concurrent connections consumes precious RAM.
  • Privacy Cost: Inspecting encrypted data or DNS payloads can compromise user confidentiality.

Furthermore, standard classification models often treat all errors equally. In a security context, misclassifying a smart camera as a smart lightbulb is far more dangerous than misclassifying it as a different brand of camera. This research addresses these challenges by optimizing for "Cyber Risk" under a resource budget.

Methodology

The authors frame feature selection as a combinatorial optimization problem aimed at minimizing a Risk Score without exceeding a cost budget ($\lambda$).

The Risk Score and Loss Matrix

To account for the varying impact of errors, the framework utilizes a Loss Matrix. This matrix allows administrators to assign higher weights to high-risk misclassifications. The total risk is calculated as the sum of the probabilities of specific errors multiplied by their respective loss values, ensuring the model is "safer" even if raw accuracy is slightly lower.

Cross-Entropy (CE) Optimization

Because brute-force searching for the optimal feature subset is computationally infeasible for large feature sets, the paper proposes a stochastic Cross-Entropy algorithm. Unlike greedy heuristics, which can suffer from performance "spikes" when forced to drop informative features to meet a budget, the CE approach evaluates feature sets holistically. It iteratively refines the probability of selecting specific features to converge on a near-optimal solution.

Experimental Setup

The framework was evaluated using real-world network traffic from 15 distinct IoT devices, including smart speakers, cameras, and sockets from various manufacturers.

Table 1: IoT devices and session counts used in the classification dataset.

The researchers extracted 111 distinct features from this traffic and compared the CE algorithm against three greedy heuristics: Cost-based (CGA), Risk-based (RGA), and Value-based (VGA).

Results and Discussion

The experimental results highlight the efficiency and effectiveness of the proposed framework.

Classifier Performance

The choice of underlying machine learning model significantly impacts performance. The study found that Decision Trees and Random Forests provided the most robust results for this task.

Figure 1: F1 score performance of various classifiers across different feature set sizes.

Key Insights

  • Optimization Stability: The CE algorithm consistently outperformed greedy methods, particularly under tight budgets. It avoided the "greedy spikes" where simple heuristics often fail by making suboptimal local choices to fit the budget.
  • Diminishing Returns: The data suggests that after selecting approximately 60-70 features, the marginal benefit of adding more data to reduce risk becomes negligible, confirming that better features are more important than more data.
  • Computational Efficiency: While brute-force methods scale exponentially, the CE algorithm's complexity grows linearly with iterations, making it viable for real-world deployment on routers and gateways.

Conclusion

This research demonstrates that cost-aware feature selection is essential for practical IoT security. By balancing classification accuracy with resource constraints and misclassification risks, the proposed framework provides a scalable solution for device identification. Future work will focus on automating privacy cost assignments and testing the system in dynamic network environments where resource availability fluctuates.

Summary updated

Abstract

Classification of IoT devices into different types is of paramount importance, from multiple perspectives, including security and privacy aspects. Recent works have explored machine learning techniques for fingerprinting (or classifying) IoT devices, with promising results. However, existing works have assumed that the features used for building the machine learning models are readily available or can be easily extracted from the network traffic; in other words, they do not consider the costs associated with feature extraction. In this work, we take a more realistic approach, and argue that feature extraction has a cost, and the costs are different for different features. We also take a step forward from the current practice of considering the misclassification loss as a binary value, and make a case for different losses based on the misclassification performance. Thereby, and more importantly, we introduce the notion of risk for IoT device classification. We define and formulate the problem of cost-aware IoT device classification. This being a combinatorial optimization problem, we develop a novel algorithm to solve it in a fast and effective way using the Cross-Entropy (CE) based stochastic optimization technique. Using traffic of real devices, we demonstrate the capability of the CE based algorithm in selecting features with minimal risk of misclassification while keeping the cost for feature extraction within a specified limit.
BibTeX
@article{chakraborty2020cost,
  author = {Biswadeep Chakraborty and Dinil Mon Divakaran and Ido Nevat and Gareth W. Peters and Mohan Gurusamy},
  title = {{Cost-aware Feature Selection for IoT Device Classification}},
  journal = {IEEE Internet of Things Journal},
  year = {2020},
  url = {https://openreview.net/forum?id=THMX6Tq8yg},
}
Figure