Two-Stage Resource Allocation in Reconfigurable Intelligent Surface Assisted Hybrid Networks via Multi-Player Bandits

Jingwen Tong, Hongliang Zhang, Liqun Fu, Amir Leshem, Han ZhuIEEE Trans. Communications 2022

Abstract

This paper considers a resource allocation problem where several Internet-of-Things (IoT) devices send data to a base station (BS) with or without the help of the reconfigurable intelligent surface (RIS) assisted cellular network. The objective is to maximize the sum rate of all IoT devices by finding the optimal RIS and spreading factor (SF) for each device. Since these IoT devices lack prior information on the RISs or the channel state information (CSI), a distributed resource allocation framework with low complexity and learning features is required to achieve this goal. Therefore, we model this problem as a two-stage multi-player multi-armed bandit (MPMAB) framework to learn the optimal RIS and SF sequentially. Then, we put forth an exploration and exploitation boosting (E2Boost) algorithm to solve this two-stage MPMAB problem by combining the $\epsilon$-greedy algorithm, Thompson sampling (TS) algorithm, and non-cooperation game method. We derive an upper regret bound for the proposed algorithm, i.e., $\mathcal{O}(\log^{1+\delta}_2 T)$, increasing logarithmically with the time horizon $T$. Numerical results show that the E2Boost algorithm has the best performance among the existing methods and exhibits a fast convergence rate. More importantly, the proposed algorithm is not sensitive to the number of combinations of the RISs and SFs thanks to the two-stage allocation mechanism, which can benefit high-density networks.

Introduction

In the next generation of cellular networks, Reconfigurable Intelligent Surfaces (RIS) are expected to significantly boost signal quality by reflecting electromagnetic waves toward users. However, integrating low-power Internet-of-Things (IoT) devices into these networks presents a significant challenge. IoT devices often operate in "blind" environments where they lack exact Channel State Information (CSI) and do not know the phase shifts of the RIS, which are typically controlled for primary cellular users. Furthermore, these devices cannot communicate with each other to coordinate, creating a massive distributed resource allocation problem.

A typical RIS-assisted hybrid uplink network where IoT devices must choose between direct paths and RIS-reflected paths.

Problem Formulation

Each IoT device must autonomously decide on two critical parameters: which RIS to utilize (to avoid collisions with other devices) and which Spreading Factor (SF) to apply (to balance data rate against transmission reliability). Traditional optimization techniques are unsuitable here due to the distributed nature of the network and the scarcity of information. Moreover, the search space—representing every possible combination of RIS and SF—is enormous, leading to slow learning in high-density networks.

The E2Boost Framework

To address these challenges, the authors propose a Two-Stage Multi-Player Multi-Armed Bandit (MPMAB) framework. The core innovation is the decoupling of the decision process: instead of searching through all combinations of RIS and SF simultaneously, the problem is split into two sequential stages:

  1. Stage 1: Identify the optimal RIS for each device.
  2. Stage 2: Identify the optimal SF for the chosen RIS.

This decoupling reduces the complexity from a multiplicative problem ($K \times M$) to an additive one ($K + M$), where $K$ is the number of RISs and $M$ is the number of SFs.

The hierarchical structure of the E2Boost algorithm, showing the transition from finding the best RIS to finding the best data rate across epochs and phases.

The E2Boost Algorithm

The proposed Exploration and Exploitation Boosting (E2Boost) algorithm operates in epochs, each consisting of three distinct phases:

  • $\epsilon$-greedy EE Phase: Devices explore different RISs. The authors utilize the Wasserstein distance to adaptively tune the exploration rate $\epsilon$. As the system converges, the algorithm automatically reduces exploration to focus on the best-performing paths.
  • Non-cooperation Game Phase: Devices engage in a distributed game to settle on a stable RIS assignment. This phase minimizes collisions purely through local observation of rewards and timeouts, requiring zero signaling overhead between devices.
  • Thompson Sampling (TS) EE Phase: Once a RIS is targeted, the device employs Thompson Sampling—a Bayesian approach—to determine the optimal data rate based on simple success/failure feedback from the base station.

Key Results and Discussion

The efficacy of E2Boost was validated through extensive numerical simulations modeled after 3GPP standards for Urban Macro (UMa) scenarios. Key findings include:

  • Theoretical Performance: The authors mathematically proved an upper regret bound of $\mathcal{O}(\log_2^{1+\delta} T)$, indicating that the algorithm's "mistakes" decrease rapidly over time.
  • Efficiency and Scalability: E2Boost is approximately $M$ times more efficient than previous state-of-the-art methods like the "Game of Thrones" (GoT) algorithm. Its performance remains robust even as the number of RIS/SF combinations increases.
  • Throughput: In simulations with 3 devices and 3 RISs, E2Boost achieved an average throughput of 2.3859 Mbps, nearing the centralized optimal solution of 2.4315 Mbps.

Conclusion

E2Boost provides a scalable, distributed solution for resource allocation in RIS-assisted IoT networks. By decoupling the learning process and employing adaptive exploration, it achieves near-optimal performance with significantly lower complexity than existing methods. Future work may explore frequency reuse and simultaneous resource sharing between IoT devices and primary cellular users.

Summary updated

Abstract

This paper considers a resource allocation problem where several Internet-of-Things (IoT) devices send data to a base station (BS) with or without the help of the reconfigurable intelligent surface (RIS) assisted cellular network. The objective is to maximize the sum rate of all IoT devices by finding the optimal RIS and spreading factor (SF) for each device. Since these IoT devices lack prior information on the RISs or the channel state information (CSI), a distributed resource allocation framework with low complexity and learning features is required to achieve this goal. Therefore, we model this problem as a two-stage multi-player multi-armed bandit (MPMAB) framework to learn the optimal RIS and SF sequentially. Then, we put forth an exploration and exploitation boosting (E2Boost) algorithm to solve this two-stage MPMAB problem by combining the $ε$-greedy algorithm, Thompson sampling (TS) algorithm, and non-cooperation game method. We derive an upper regret bound for the proposed algorithm, i.e., $\mathcal{O}(\log^{1+δ}_2 T)$, increasing logarithmically with the time horizon $T$. Numerical results show that the E2Boost algorithm has the best performance among the existing methods and exhibits a fast convergence rate. More importantly, the proposed algorithm is not sensitive to the number of combinations of the RISs and SFs thanks to the two-stage allocation mechanism, which can benefit high-density networks.
BibTeX
@article{tong2022two,
  author = {Jingwen Tong and Hongliang Zhang and Liqun Fu and Amir Leshem and Han Zhu},
  title = {{Two-Stage Resource Allocation in Reconfigurable Intelligent Surface Assisted Hybrid Networks via Multi-Player Bandits}},
  journal = {IEEE Trans. Communications},
  year = {2022},
}
Figure