QoE-Oriented Resource Allocation for 360-degree Video Transmission over Heterogeneous Networks

Wei Huang, Lianghui Ding, Hung-Yu Wei, Jenq-Neng Hwang, Yiling Xu, Wenjun ZhangarXiv 2018 · OpenReview

Abstract

Immersive media streaming, especially virtual reality (VR)/360-degree video streaming which is very bandwidth demanding, has become more and more popular due to the rapid growth of the multimedia and networking deployments. To better explore the usage of resource and achieve better quality of experience (QoE) perceived by users, this paper develops an application-layer scheme to jointly exploit the available bandwidth from the LTE and Wi-Fi networks in 360-degree video streaming. This newly proposed scheme and the corresponding solution algorithms utilize the saliency of video, prediction of users' view and the status information of users to obtain an optimal association of the users with different Wi-Fi access points (APs) for maximizing the system's utility. Besides, a novel buffer strategy is proposed to mitigate the influence of short-time prediction problem for transmitting 360-degree videos in time-varying networks. The promising performance and low complexity of the proposed scheme and algorithms are validated in simulations with various 360-degree videos.

Introduction

Streaming 360-degree video is a "bandwidth hog." To provide a truly immersive experience, resolutions often need to exceed 4K, leading to a projected 20-fold increase in VR traffic. While "tile-based" streaming—dividing the video into spatial segments and only sending what the user sees—helps, it faces three major hurdles: prediction uncertainty, multi-user congestion, and network heterogeneity. Modern devices can use LTE and Wi-Fi simultaneously (multi-RAT), but coordinating these resources to maximize Quality of Experience (QoE) is a complex optimization problem.

System Architecture and Methodology

The authors propose a novel resource allocation framework that integrates video saliency, field-of-view (FoV) prediction, and a hierarchical buffer strategy. The system architecture involves a centralized controller that manages resources across LTE and Wi-Fi networks.

System architecture for 360-degree video transmission over heterogeneous networks.

The methodology consists of three main components:

  • QoE-Driven Utility Model: Instead of just looking at bitrates, the model combines saliency and FoV probability. It includes a "weakest link" penalty—if one tile in the user's view is low quality, it drags down the entire experience.
  • Multi-RAT Resource Allocation: The problem of assigning users to Wi-Fi APs and allocating bitrates across LTE/Wi-Fi is formulated as a mixed-integer NP-hard problem. The authors propose three algorithms: a Greedy Algorithm, a Heuristic with an $\ell_1$-norm penalty function to encourage sparsity in AP association, and a Decomposition Algorithm for faster execution.
  • Hierarchical Buffer Strategy: To handle the trade-off between buffer stability and FoV prediction accuracy, two thresholds are used. A short buffer ($B_1$) stores high-quality, FoV-specific tiles, while a longer buffer ($B_2$) stores low-quality versions of the entire frame as a "safety net" against network drops or prediction errors.

Traditional video metrics often average quality, but in VR, a single blurry tile in the field of vision is highly distracting. The proposed utility model specifically penalizes the lowest-quality tile in the FoV. Furthermore, when bandwidth is tight, the system prioritizes tiles with higher saliency weights.

Saliency detection and tile-based weight assignment for 360-degree video.

As shown in the figure above, saliency detection helps identify the most "interesting" parts of the video, which are then given higher weights during resource allocation. This ensures that even if the whole FoV cannot be high-definition, the most important areas remain clear.

Experimental Evaluation and Results

The scheme was evaluated using 18 distinct 360-degree videos (e.g., AerialCity, DrivingInCity) in NS-3 simulations. Key findings include:

  • Near-Optimal Performance: The heuristic algorithm achieved results close to the theoretical maximum with significantly lower computational complexity.
  • Congestion Resilience: In crowded scenarios (15+ users), the centralized scheme maintained higher total utility compared to decentralized methods where LTE and Wi-Fi networks operate in silos.
  • Buffer Stability: The hierarchical strategy prevented "frozen" screens during network fluctuations, maintaining high average utility where standard strategies failed.

Conclusion

This paper presents a robust framework for 360-degree video transmission. By leveraging multi-RAT resources and a QoE-oriented utility model, the system significantly improves user experience. Future work will focus on dynamic buffer threshold adjustment and extending the framework to multicast scenarios for popular live VR events.

Summary updated

Abstract

Immersive media streaming, especially virtual reality (VR)/360-degree video streaming which is very bandwidth demanding, has become more and more popular due to the rapid growth of the multimedia and networking deployments. To better explore the usage of resource and achieve better quality of experience (QoE) perceived by users, this paper develops an application-layer scheme to jointly exploit the available bandwidth from the LTE and Wi-Fi networks in 360-degree video streaming. This newly proposed scheme and the corresponding solution algorithms utilize the saliency of video, prediction of users' view and the status information of users to obtain an optimal association of the users with different Wi-Fi access points (APs) for maximizing the system's utility. Besides, a novel buffer strategy is proposed to mitigate the influence of short-time prediction problem for transmitting 360-degree videos in time-varying networks. The promising performance and low complexity of the proposed scheme and algorithms are validated in simulations with various 360-degree videos.
BibTeX
@misc{huang2018qoe,
  author = {Wei Huang and Lianghui Ding and Hung-Yu Wei and Jenq-Neng Hwang and Yiling Xu and Wenjun Zhang},
  title = {{QoE-Oriented Resource Allocation for 360-degree Video Transmission over Heterogeneous Networks}},
  year = {2018},
  eprint = {1803.07789},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1803.07789},
}
Figure