Massive multi-player multi-armed bandits for IoT networks: An application on LoRa networks

Hiba Dakdouk, Raphaël Féraud, Nadège Varsier, Patrick Maillé, Romain LarocheAd Hoc Networks 2023 · DOI

TL;DR

This paper tackles the challenge of optimizing energy consumption and packet delivery in massive IoT networks, specifically using LoRaWAN. They propose novel multi-player multi-armed bandit algorithms (DORG and DOFG) that outperform existing methods like LoRaWAN's Adaptive Data Rate (ADR) by intelligently managing device transmission parameters.

What Problem Does This Paper Address?

The proliferation of Internet of Things (IoT) devices, particularly in the context of Industry 4.0, presents significant challenges in terms of network reliability and energy efficiency. LoRaWAN, a popular Low Power Wide Area Network (LPWAN) technology, is widely deployed for its long-range capabilities. However, the shared, unlicensed spectrum in LPWANs leads to increased interference and packet collisions, especially with a massive number of devices. Existing solutions like LoRaWAN's Adaptive Data Rate (ADR) algorithm are heuristic and treat devices individually, failing to optimize for the collective network performance. This paper aims to address this by formulating the problem as a "massive multi-player multi-armed bandit" (MP-MAB) problem, where each IoT device is a player, and each communication channel is an arm. The goal is to maximize successful transmissions while minimizing energy consumption and ensuring fairness among devices.

Methods

The authors propose two novel algorithms based on the multi-player multi-armed bandit framework:

  • DORG (Decreasing-Order-Reward-Greedy): This algorithm aims to maximize the number of successful transmissions. It prioritizes assigning devices to channels that offer the highest expected reward, considering the probabilities of other devices transmitting.
  • DOFG (Decreasing-Order-Fair-Greedy): This algorithm focuses on ensuring fairness among devices in terms of successful transmission rates, while still aiming for good overall performance.

They also introduce a collaborative exploration approach. Since individual IoT devices cannot directly observe collisions, they propose a decentralized method where devices share their learned information (e.g., estimated channel rewards) to collectively build a model of the network. This learned model is then used by the DORG and DOFG policies for exploitation. This "explore-then-exploit" strategy is designed to be efficient and suitable for resource-constrained IoT devices.

Datasets

Not discussed in the paper. The authors use a LoRa network simulator for their experiments.

Key Insights and Ideas

  • Massive MP-MAB is a powerful framework for IoT optimization: The paper demonstrates that framing IoT network optimization as a massive multi-player multi-armed bandit problem can lead to significant improvements over existing methods.
  • Collaboration is key for efficiency: A decentralized collaborative exploration approach allows devices to learn network parameters more efficiently than individual selfish learning.
  • Fairness and performance are often a trade-off, but can be managed: The DORG and DOFG algorithms show that it's possible to achieve high performance while also ensuring a degree of fairness among devices, which is crucial for cooperative network behavior.
  • LoRaWAN's ADR is suboptimal for massive deployments: The experimental results clearly show that the current ADR mechanism is outperformed by the proposed MAB-based approaches in terms of both energy efficiency and packet delivery.

Key Results

  • Significant improvement over ADR: The proposed DORG and DOFG algorithms, when applied to a LoRa network simulator, significantly outperform the standard ADR algorithm in terms of energy consumption and packet losses.
  • DORG maximizes successful transmissions: DORG consistently achieves higher successful communication rates.
  • DOFG ensures fairness: DOFG provides a much higher level of fairness between devices, even at the cost of a slight reduction in overall performance compared to DORG.
  • Collaborative exploration is faster: The collaborative exploration phase is shown to be orders of magnitude faster than selfish exploration methods.
  • Robustness to non-stationarity: The algorithms show good performance even when some nodes use the ADR mechanism, introducing non-stationarity into the network.

Limitations and Future Work

  • Theoretical vs. Real-world gap: The authors acknowledge that their theoretical model makes some simplifying assumptions (e.g., orthogonal channels, same arm characteristics for all players) that differ from the real LoRa environment.
  • Non-orthogonal channels and varying rewards: Future work will focus on adapting the DORG and DOFG policies to handle non-orthogonal channels and scenarios where arm rewards differ across players.
  • Non-stationary environments: Extending the algorithms to better handle dynamic environments where network conditions change rapidly is a key future direction.
  • Sub-slotting for improved ALOHA: The paper suggests exploring sub-slotting in slotted-ALOHA to better manage varying packet transmission times and reduce collisions.
  • Ethical considerations and malicious players: For real-world deployment, future work needs to consider weighted fairness based on device purpose and protection against malicious players who might lie about their activity or channel rewards.

Summary updated

BibTeX
@article{dakdouk2023massive,
  author = {Hiba Dakdouk and Raphaël Féraud and Nadège Varsier and Patrick Maillé and Romain Laroche},
  title = {{Massive multi-player multi-armed bandits for IoT networks: An application on LoRa networks}},
  journal = {Ad Hoc Networks},
  year = {2023},
  doi = {10.1016/j.adhoc.2023.103283},
}
Figure