Abstract
In dense Wireless Local Area Networks (WLANs), high-density Access Points (APs) bring severe interference that seriously affects the experience of users, resulting in lower throughput and poor connection quality. Due to the heavy computation workload raised by the sizable networking systems and the difficulty in estimating instantaneous Channel State Information (CSI), existing works are hard to solve interference problem. In this paper, we propose a Joint Power control and Channel allocation based on Reinforcement Learning (JPCRL) algorithm combining with statistical CSI to reduce interference adaptively. Firstly, we analyze the correlation between transmit power and channel, and formulate the interference optimization as a Mixed Integer Nonlinear Programming (MINLP) problem. Secondly, we use the statistical CSI method to take the power and channel state as the state and action space, the overall throughput increment as the reward function of Q-learning, and obtain the optimal joint optimization strategy through off-line training. Moreover, for the periodic reinforcement learning process leading to resource consumption, we design an event-driven mechanism of Q-learning, which triggers online learning to refresh the optimal policy by event-driven condition and the consumption of computing resources can be reduced. The evaluation results show that the proposed algorithm can effectively improve the throughput compared with the existing scheme.
Introduction
The proliferation of Wi-Fi-enabled devices in public spaces like stadiums, shopping malls, and office buildings has led to the deployment of high-density Wireless Local Area Networks (WLANs). While increasing the number of Access Points (APs) aims to improve coverage and capacity, it often introduces significant co-channel and adjacent-channel interference. This "gridlock" degrades user experience, leading to dropped connections and reduced throughput.
Managing this interference requires optimizing two critical parameters: transmit power and channel allocation. However, solving this jointly is a Mixed Integer Nonlinear Programming (MINLP) problem, which is computationally expensive and typically NP-hard. Furthermore, traditional methods often rely on instantaneous Channel State Information (CSI), which is difficult and resource-intensive to acquire in dynamic environments.
Methodology: The JPCRL Algorithm
To address these challenges, we introduce the Joint Power control and Channel allocation based on Reinforcement Learning (JPCRL) algorithm. JPCRL leverages the adaptive nature of reinforcement learning to navigate the complex trade-offs between power levels and channel selections.
System Architecture
The proposed system utilizes a centralized controller that manages multiple APs and their associated Stations (STAs). The controller monitors network conditions and dictates optimal configurations to mitigate interference across the deployment.
Key Technical Components
- Q-Learning Framework: The optimization problem is modeled as a Markov Decision Process. The controller (agent) observes the network state—current power and channel settings—takes actions to modify these settings, and receives a reward based on the resulting increase in total system throughput.
- Statistical CSI: Instead of chasing volatile instantaneous signal changes, the model uses statistical CSI. This approach provides a more stable representation of the wireless environment, reducing the overhead of frequent measurements and making the learning process more robust against short-term noise.
- Event-Driven Mechanism: To minimize computational overhead, the authors implemented an event-driven trigger. Instead of retraining the model at fixed intervals, the system only initiates a new learning phase when a significant change in the network environment (e.g., a shift in user distribution or topology) is detected beyond a predefined threshold ($\delta$).
Experimental Evaluation
The performance of JPCRL was evaluated using MATLAB simulations of a dense WLAN scenario covering a 100m x 100m area with 15 APs and varying user densities ranging from 10 to 90 STAs.
Key Results
- Throughput Enhancement: JPCRL achieved a 1.7x increase in total throughput compared to power control-only schemes and a 16% improvement over traditional joint power control (JPC) methods.
- Interference Mitigation: The algorithm reduced total system interference by approximately 80% compared to basic power control strategies.
- Scalability: The algorithm maintained superior average throughput per user even as the network density increased to 90 users, showcasing its effectiveness in high-stress environments.
Conclusion
JPCRL offers a promising solution for interference management in dense WLANs by combining reinforcement learning with an efficient event-driven mechanism. By utilizing statistical CSI and joint optimization, it significantly boosts network performance while maintaining manageable computational costs. Future work will focus on decentralizing the control mechanism and validating the algorithm on real-world Wi-Fi 6/7 hardware to account for non-ideal physical layer behaviors.
Summary updated