Abstract
Integrated Sensing and Communication (ISAC) is a cornerstone of 6G technology, yet the deployment of flexible antenna systems remains a challenge due to cost and complexity. This paper proposes a joint optimization framework for ISAC systems using low-cost "pinching antennas." We introduce a Maximum Entropy-based Reinforcement Learning (MERL) algorithm to optimize antenna positions and transmit power simultaneously. By maximizing entropy alongside cumulative rewards, the MERL agent avoids local optima and develops robust policies. Numerical results show that MERL improves normalized data rates by up to 44.4% and sensing accuracy by 16.7% compared to standard reinforcement learning benchmarks.
Introduction
Integrated Sensing and Communication (ISAC) represents a paradigm shift for 6G, enabling the sharing of hardware and spectrum between radar-like sensing and high-speed data transmission. A significant challenge in dynamic ISAC environments is the limitation of fixed-position antennas. While flexible solutions like fluid or movable antennas exist, they are often prohibitively expensive or limited in range.
This research focuses on pinching antennas, a cost-effective alternative where dielectric materials are placed on a waveguide to create Line-of-Sight (LoS) links. The core problem is the joint optimization of antenna positions and transmit power to maximize communication throughput without compromising sensing accuracy. This results in a complex, non-convex optimization problem that traditional methods struggle to resolve efficiently.
System Model and Methods
The proposed system utilizes a waveguide with multiple pinching points that act as antennas. By moving these points, the system can adapt its radiation pattern to the environment.
To solve the optimization challenge, the authors developed a Maximum Entropy-based Reinforcement Learning (MERL) framework. Unlike standard Reinforcement Learning (RL), which focuses solely on reward maximization, MERL incorporates an entropy term into its objective function. This encourages the agent to explore a broader range of actions, preventing premature convergence to local optima and enhancing policy robustness.
The state space includes antenna positions, user and target locations, and remaining energy. The action space consists of antenna displacements and power allocation per user. To ensure physical feasibility, the model enforces a minimum distance between pinching points to account for antenna coupling.
Experimental Setup
The performance of the MERL algorithm was evaluated through numerical simulations in a 150m × 150m area with 6 mobile users and 3 pinching antennas operating at a 28 GHz carrier frequency. The proposed method was benchmarked against three schemes:
- DDPG (Deep Deterministic Policy Gradient)
- TD3 (Twin Delayed Deep Deterministic Policy Gradient)
- Random Baseline
Results and Discussion
The simulation results highlight the clear advantages of the MERL approach. The algorithm achieved a higher and more stable reward plateau compared to its predecessors.
Key performance metrics include:
- Data Rate Improvement: MERL improved the normalized total data rate by 20.3% over TD3 and 44.4% over DDPG.
- Sensing Accuracy: The normalized sensing Signal-to-Noise Ratio (SNR) improved by 16.7% compared to TD3.
- Stability and Robustness: Training curves for MERL exhibited significantly less variance, and the algorithm remained high-performing across various learning rates, indicating low sensitivity to hyperparameter tuning.
Conclusion
This paper demonstrates that pinching antennas combined with a Maximum Entropy-based Reinforcement Learning framework offer a high-performance, low-cost solution for dynamic ISAC systems. By balancing exploration and exploitation, the MERL algorithm effectively navigates the complex optimization landscape of joint antenna positioning and power allocation. Future research will explore the integration of Non-Orthogonal Multiple Access (NOMA) and address the mechanical speed constraints of physical antenna movement.
Summary updated