Rlaph: a lightweight and dynamic proactive defense method in cloud-edge collaboration

Yingbo Li, Jie Yuan, Frank Jiang, Rui Qian, Xiang Liu, Xinghai Wei, Xiaoyong LiCybersecurity 2025 · DOI

Introduction

In the evolving landscape of cybersecurity, the "attacker’s advantage" remains a significant challenge: defenders must secure every potential vulnerability, while an attacker needs only a single entry point. This vulnerability is particularly acute in cloud-edge collaboration environments. Decentralized edge devices, such as IoT sensors or smart city infrastructure, often lack robust security, making them ideal targets for initial compromise and subsequent lateral movement into core cloud networks.

Traditional defense mechanisms are primarily passive, reacting only after a threat is detected. Moving Target Defense (MTD) shifts this paradigm by making the network a "moving target." By constantly changing IP addresses and ports, MTD ensures that an attacker’s reconnaissance data becomes obsolete almost immediately. However, existing MTD strategies often suffer from rigidity, one-dimensionality, or a lack of historical context, leading to predictable patterns or performance degradation.

The RLAPH Framework

To address these limitations, the authors propose RLAPH (Reinforcement Learning-based Address and Port Hopping). RLAPH treats proactive defense as an optimization problem, utilizing Reinforcement Learning (RL) to dynamically adjust the network's attack surface.

The RLAPH system architecture integrates an Actor-Critic RL module with an SDN controller to manage network mutations.

The system is built on several key technical pillars:

  1. SMDP Modeling: The network state is modeled as a Semi-Markov Decision Process (SMDP), allowing the system to account for the time-varying nature of network events and transitions.
  2. Multi-Dimensional Hopping: Unlike single-tactic systems, RLAPH can choose to hop IPs, hop ports, do both, or remain stationary, depending on the perceived threat level and current network state.
  3. SDN Integration: The system operates within a Software-Defined Networking (SDN) controller. This allows for transparent remapping of virtual-to-real IPs and ports, ensuring that legitimate users experience no service interruptions while attackers are thwarted.

Neural Network Architecture and Temporal Features

A core innovation of RLAPH is its use of historical data to inform defense decisions. The system employs an improved Actor-Critic (AC) architecture to balance security and overhead.

The Actor-Critic network uses 1D-CNNs to extract temporal features from past scan hits and mutation history.

To capture attacker behavior over time, RLAPH utilizes 1D-Convolutional Neural Networks (1D-CNNs). These networks process sequences of past scan hits and mutation frequencies (the state $s_t$). The "Actor" network determines the optimal hopping action, while the "Critic" network evaluates the action's quality based on a custom reward function. This reward function is specifically designed to penalize excessive resource consumption, forcing the agent to learn the most efficient movement strategy.

Evaluation and Key Results

The researchers evaluated RLAPH in a simulated environment using OpenAI Gym, featuring a topology of 5 switches and 30 hosts. The simulation included common reconnaissance strategies like Local Priority Scanning and Sequential Scanning.

  • Superior Defense Performance: RLAPH achieved a composite reward score approximately 22% higher than baseline strategies, such as fixed-time or random hopping.
  • The Importance of History: Ablation studies revealed that looking at the last 6 time slots ($k=6$) provided the optimal balance between security stability and computational latency.
  • High Efficiency: Despite the use of deep learning, the model's inference time was recorded at just 0.137 ms, making it lightweight enough for real-time deployment in high-speed environments.

Conclusion

RLAPH demonstrates that a reinforcement learning approach to MTD can significantly enhance network security while maintaining operational efficiency. By leveraging historical attack patterns and multi-dimensional hopping, it effectively disrupts the reconnaissance phase of cyberattacks. Future work will focus on expanding the system's capabilities to counter DDoS attacks and improving retraining speeds to adapt to rapidly evolving threats.

Summary updated

Abstract

Abstract In cloud-edge collaboration scenarios, attackers pose significant security risks by compromising computational nodes and using them to infiltrate other nodes and networks. Ensuring the security of cloud-edge collaboration is crucial for protecting sensitive data, preventing disruptions to critical services, and safeguarding infrastructure in increasingly interconnected and digitized societies. Traditional passive defense mechanisms are often inadequate in dealing with the complex and dynamic nature of modern network threats. In recent years, Moving Target Defense (MTD) has become an important research direction, disrupting adversaries’ reconnaissance and exploitation phases by dynamically shuffling the attack surface. However, existing MTD strategies have some shortcomings, such as single-dimensional movement strategies, poor flexibility and a lack of historical information analysis. To overcome these challenges, we propose a reinforcement learning-based approach for host address and port hopping (RLAPH). First, the approach strengthens system security through coordinated decision-making across IP address and port, leveraging both historical data and current information to make accurate and adaptive decisions. Second, a reward function is carefully designed to balance the trade-off between system overhead and security. Finally, validation experiments conducted in a simulated environment show that the proposed method effectively enhances defense performance while minimizing system overhead, highlighting its robustness and applicability.
BibTeX
@inproceedings{li2025rlaph,
  author = {Yingbo Li and Jie Yuan and Frank Jiang and Rui Qian and Xiang Liu and Xinghai Wei and Xiaoyong Li},
  title = {{Rlaph: a lightweight and dynamic proactive defense method in cloud-edge collaboration}},
  booktitle = {Cybersecurity},
  year = {2025},
  doi = {10.1186/s42400-025-00392-3},
}
Figure