The proliferation of Internet of Things (IoT) devices has created a massive attack surface for rapidly evolving malware. Traditional centralized security measures often suffer from high latency and resource constraints. This research, published in the IEEE Internet of Things Journal, proposes a decentralized approach using Edge Intelligence-enabled IoT (EIIoT) to autonomously suppress malware spread through advanced reinforcement learning.
The Challenge of IoT Malware Suppression
Modern IoT malware evolves "sharply," making static patching and manual analysis ineffective. Furthermore, these environments are characterized by "incomplete information," where neither the defender nor the attacker has full visibility into the other's state. To address this, the authors propose moving the defensive "brain" from the cloud to edge nodes, enabling localized, low-latency autonomous responses.
Stochastic Games and Edge Intelligence
The core of the proposed solution is the Stochastic Games-based Malware Spread-Suppression (SGMSS) model. This mathematical framework treats the interaction between malware and edge defenders as a multi-agent, dynamic game with probabilistic state transitions. The goal is to find a Nash equilibrium—a stable strategy where neither party can improve their outcome by unilaterally changing their approach.
To solve this game, the researchers implemented three variants of Deep Q-Networks (DQN) designed to overcome common pitfalls like value overestimation:
- DDQMS (Double DQN for Malware Spread): Decouples action selection from value estimation using two separate networks, leading to more accurate Q-values.
- D2QMS (Dueling DQN for Malware Spread): Splits the network's final layer to independently estimate state value and action advantage, improving learning in states where actions have delayed impacts.
- D3QMS (Dueling Double DQN for Malware Spread): A hybrid architecture combining both Double and Dueling DQN features for maximum stability and performance.
Experimental Framework
The algorithms were evaluated using idsgame, an OpenAI Gym environment for modeling stochastic games in cybersecurity. The environment simulates a Partially Observable Markov Decision Process (POMDP). In these simulations, malware followed a random spread policy while edge nodes were trained using a Fuzzy Neural Network (FNN) with two hidden layers and ReLU activation.
Key Insights and Performance Results
The study yielded several critical findings for autonomous network defense:
- Superiority of D3QMS: The hybrid D3QMS algorithm emerged as the top performer, achieving the highest cumulative rewards and the fastest reduction in average episode loss (dropping from 0.0042 to 0.0008).
- Effective Suppression: All three proposed algorithms successfully maintained the malware spread rate below 0.15, demonstrating the viability of autonomous edge-based defense.
- The "Short-Sighted" Advantage: Counterintuitively, a lower discount factor ($\gamma = 0.85$) outperformed the standard $\gamma = 0.99$. In highly volatile malware environments, prioritizing immediate suppression results leads to more robust long-term defense.
- Optimal Hyperparameters: The researchers identified a learning rate ($\alpha$) of 4e-4 and a discount factor of 0.85 as the "sweet spot" for training these agents.
Future Directions
While the results demonstrate the power of edge intelligence, the authors note that future work should test these defenses against "intelligent" attackers using their own reinforcement learning strategies. Additionally, further research is needed to quantify the computational overhead on resource-constrained IoT hardware and to evaluate performance in larger, more complex network topologies.
Summary updated