Introduction
As the telecommunications industry moves toward 6G, Unmanned Aerial Vehicles (UAVs) are increasingly deployed as aerial access points to provide network services in dense urban environments or disaster zones. While UAVs offer high mobility and flexibility, their air-to-ground communication links are predominantly Line-of-Sight (LoS), making them highly susceptible to interception by ground-based eavesdroppers. Traditional cryptographic security measures can be computationally prohibitive for resource-constrained IoT devices. Consequently, Physical Layer Security (PLS) has emerged as a promising alternative, leveraging the physical properties of the wireless channel to ensure secure transmission.
System Model and Problem Formulation
To enhance PLS, this paper proposes a UAV-ground secure communication system assisted by a Reconfigurable Intelligent Surface (RIS). The RIS, composed of many passive reflecting elements, is deployed on building facades to intelligently reflect signals, thereby improving the signal quality for legitimate users while degrading it for eavesdroppers.
The research objective is to maximize the Sum Secrecy Rate (SSR) and Secrecy Energy Efficiency (SEE) of the system. This involves the joint optimization of several coupled variables: the UAV's flight trajectory, active beamforming at the UAV, passive beamforming at the RIS, and transmission power. Due to the dynamic nature of the environment and the non-convexity of the optimization problem, traditional methods struggle to provide real-time solutions.
The MATD3-EP Algorithm
The authors introduce the Multi-Agent Twin Delayed Deep Deterministic Policy Gradient with Energy Penalty (MATD3-EP) algorithm. This framework addresses the optimization challenge by decomposing it into manageable subtasks handled by multiple agents.
The algorithm incorporates three key innovations:
- Multi-Agent Cooperation: Instead of a single complex controller, the system uses multiple agents that share a global reward. This decoupling prevents "inter-task interference," where optimizing one parameter (like trajectory) might negatively impact another (like beamforming).
- Twin Delayed DDPG (TD3): TD3 improves upon standard DDPG by using two critic networks to estimate the Q-value, taking the minimum of the two to prevent overestimation bias. This results in significantly more stable training and better convergence.
- Energy Penalty (EP) Mechanism: To address the limited battery life of UAVs, the reward function includes a dynamic energy penalty. This penalty scales based on the current secrecy performance, forcing the agents to find a balance between maximizing security and minimizing propulsion and transmission energy.
Performance Evaluation
The proposed MATD3-EP algorithm was tested in a simulated 3D environment using the 3D Spatial Variation (SV) channel model at a 28 GHz carrier frequency.
The simulation results demonstrate that MATD3-EP significantly outperforms benchmark strategies:
- Secrecy Performance: The algorithm achieved a steady SSR of approximately 5 bit/s/Hz, which is more stable and higher than standard MATD3 or PPO-based methods.
- Energy Efficiency: By incorporating the energy penalty, the system reduced total energy consumption to 11.01 KJ, a substantial improvement over the 14.70 KJ consumed by the D-PPO benchmark.
- Trajectory Intelligence: The UAV learned to fly in a pattern that maximizes the distance from eavesdroppers while staying within the "reflection zone" of the RIS to maintain high-quality links with legitimate users.
Conclusion and Future Directions
This work highlights the potential of combining RIS technology with multi-agent reinforcement learning to solve complex security and energy challenges in UAV-assisted networks. While the current model assumes a fixed flight altitude and perfect RIS phase shifts, future research will focus on 3D trajectory optimization, hardware-constrained RIS modeling, and scenarios with imperfect Channel State Information (CSI) to further bridge the gap between simulation and real-world deployment.
Summary updated