Liquid State Machine Learning for Resource and Cache Management in LTE-U Unmanned Aerial Vehicle (UAV) Networks

Mingzhe Chen, Walid Saad, Changchuan Yin

Abstract

In this paper, the problem of joint caching and resource allocation is investigated for a network of cache-enabled unmanned aerial vehicles (UAVs) that service wireless ground users over the LTE licensed and unlicensed (LTE-U) bands. The considered model focuses on users that can access both licensed and unlicensed bands while receiving contents from either the cache units at the UAVs directly or via content server-UAV-user links. This problem is formulated as an optimization problem which jointly incorporates user association, spectrum allocation, and content caching. To solve this problem, a distributed algorithm based on the machine learning framework of liquid state machine (LSM) is proposed. Using the proposed LSM algorithm, the cloud can predict the users' content request distribution while having only limited information on the network's and users' states. The proposed algorithm also enables the UAVs to autonomously choose the optimal resource allocation strategies that maximize the number of users with stable queues depending on the network states. Based on the users' association and content request distributions, the optimal contents that need to be cached at UAVs as well as the optimal resource allocation are derived. Simulation results using real datasets show that the proposed approach yields up to 33.3% and 50.3% gains, respectively, in terms of the number of users that have stable queues compared to two baseline algorithms: Q-learning with cache and Q-learning without cache. The results also show that LSM significantly improves the convergence time of up to 33.3% compared to conventional learning algorithms such as Q-learning.

1. Introduction

Deploying Unmanned Aerial Vehicles (UAVs) as flying base stations is a promising solution for providing on-demand wireless coverage. However, two major bottlenecks exist: limited fronthaul capacity between the cloud and UAVs, and spectrum scarcity in licensed bands. While LTE-Unlicensed (LTE-U) allows UAVs to utilize the 5GHz unlicensed band, managing interference with existing WiFi users while ensuring "queue stability" (where service rate meets or exceeds arrival rate) is complex. This paper proposes a joint optimization of user association, spectrum allocation, and content caching to maximize the number of users with stable service.

2. System Model

The network architecture consists of multiple cache-enabled UAVs serving ground users. Users can receive content either directly from local UAV caches or via the cloud-to-UAV link. The system utilizes both licensed and unlicensed bands to deliver content.

System model for the UAV-based LTE-U network with caching.

The primary objective is to optimize resource allocation to ensure that the maximum number of users maintain stable data queues despite the dynamic nature of content requests and interference in the unlicensed spectrum.

3. Proposed LSM-based Framework

The authors propose a distributed framework based on Liquid State Machines (LSM), a type of reservoir computing using spiking neural networks. Unlike standard neural networks, LSMs treat the network state as a continuous flow, where past inputs influence current decisions through a "fading memory" effect.

3.1 Content Request Prediction

The cloud uses an LSM to predict the probability distribution of user requests based on context information (e.g., age, gender, device type). This allows the system to proactively cache popular content, bypassing the fronthaul bottleneck.

3.2 Self-Organizing Resource Allocation

Each UAV acts as an autonomous agent in an LSM-based reinforcement learning loop. UAVs observe the network state and choose optimal association and spectrum strategies. To coexist with WiFi users, the UAVs employ a duty cycling method in the unlicensed band.

LTE-U frame structure and duty cycling mechanism.

As illustrated in the frame structure, the LSM learns the optimal fraction of time slots ($\vartheta$) to allocate to LTE-U users to maintain queue stability without disrupting local WiFi performance.

4. Key Results and Insights

Experimental evaluations using real-world datasets from Youku demonstrate the following advantages of the LSM approach:

  • Stability Gains: The LSM approach yielded a 33.3% gain in the number of users with stable queues compared to Q-learning with caching, and a 50.3% gain over Q-learning without caching.
  • Faster Convergence: The LSM reached convergence 33.3% faster than conventional Q-learning, a critical factor for mobile networks where states change rapidly.
  • Prediction Accuracy: The LSM predicted content request distributions with an accuracy within 10% of real-world data, outperforming Echo State Networks (ESN) by reducing mean squared error by 13.4%.

5. Conclusion

This research demonstrates that Liquid State Machines provide a robust and efficient framework for managing the complexities of UAV-based LTE-U networks. By combining temporal prediction with autonomous resource allocation, the proposed system significantly improves network stability and learning speed. Future work will investigate the impact of UAV mobility and energy constraints on these optimization strategies.

Summary updated

Abstract

In this paper, the problem of joint caching and resource allocation is investigated for a network of cache-enabled unmanned aerial vehicles (UAVs) that service wireless ground users over the LTE licensed and unlicensed (LTE-U) bands. The considered model focuses on users that can access both licensed and unlicensed bands while receiving contents from either the cache units at the UAVs directly or via content server-UAV-user links. This problem is formulated as an optimization problem which jointly incorporates user association, spectrum allocation, and content caching. To solve this problem, a distributed algorithm based on the machine learning framework of liquid state machine (LSM) is proposed. Using the proposed LSM algorithm, the cloud can predict the users' content request distribution while having only limited information on the network's and users' states. The proposed algorithm also enables the UAVs to autonomously choose the optimal resource allocation strategies that maximize the number of users with stable queues depending on the network states. Based on the users' association and content request distributions, the optimal contents that need to be cached at UAVs as well as the optimal resource allocation are derived. Simulation results using real datasets show that the proposed approach yields up to 33.3% and 50.3% gains, respectively, in terms of the number of users that have stable queues compared to two baseline algorithms: Q-learning with cache and Q-learning without cache. The results also show that LSM significantly improves the convergence time of up to 33.3% compared to conventional learning algorithms such as Q-learning.
BibTeX
@article{chen2019liquid,
  author = {Mingzhe Chen and Walid Saad and Changchuan Yin},
  title = {{Liquid State Machine Learning for Resource and Cache Management in LTE-U Unmanned Aerial Vehicle (UAV) Networks}},
  journal = {IEEE Trans. Wireless Communications},
  year = {2019},
  url = {https://openreview.net/forum?id=PEmRiII6Da_N},
}
Figure