A Deep Reinforcement Learning Approach to Balance Viewport Prediction and Video Transmission in 360° Video Streaming

Guanghui Zhang, Jing GuoIJCAI 2024

Introduction

Streaming 360° video presents a unique set of challenges compared to traditional 2D video. To optimize bandwidth, modern players employ a "tiled" approach, downloading high-quality segments only for the user's current field of view (the viewport). However, this introduces a fundamental conflict between prediction accuracy and transmission stability.

The Viewport-Transmission Dilemma

The efficiency of 360° streaming hinges on a delicate balance. To accurately predict a user's next head movement, the "prediction distance" must be minimized, which requires maintaining a small video buffer. Conversely, to ensure smooth playback and prevent rebuffering during network fluctuations, a large buffer is necessary. Existing solutions typically prioritize one over the other, resulting in either frequent stuttering or degraded visual quality due to misaligned viewport predictions.

The relationship between buffer size, prediction distance, and future segment pre-downloading.

Methods: The QUTA System

To resolve this bottleneck, we introduce QUTA, a system based on Continuous Control Deep Reinforcement Learning (CC-DRL). QUTA dynamically manages the buffer and prediction distance by controlling three critical parameters:

  1. Download Pause ($\sigma$): Periodically halting downloads to allow the buffer level to decrease.
  2. Data Rate Threshold ($\mu$): Capping the total bitrate of tiles to optimize download times.
  3. Playback Rate ($\rho$): Subtly adjusting the video speed (within a ±20% range) to consume the buffer more or less aggressively.

Long-Term-Planning (LTP) Training

Traditional DRL methods often suffer from "short-sightedness," optimizing only for the immediate next segment. QUTA utilizes a novel Long-Term-Planning (LTP) training method. By sampling sequences of transitions and calculating expected rewards across multiple future steps, the model can pursue a global optimum, ensuring stability over the entire streaming session rather than making greedy, local decisions.

Experimental Results

QUTA was evaluated using a comprehensive dataset of 4,275 real user viewport trajectories and over 60,000 network traces. The system demonstrated significant improvements in Quality-of-Experience (QoE), outperforming state-of-the-art baselines like Flare, SalientVR, and TRACK by 20.5% to 68.4%.

Performance comparison of QUTA against baseline methods across QoE, video quality, prediction accuracy, and rebuffering time.

Key findings include:

  • Superior QoE: QUTA consistently achieved higher QoE scores across various network conditions.
  • Stability: The system maintained high average video quality while minimizing rebuffering events, even in "poor" network environments.
  • Efficiency: The LTP training method allowed the model to converge to a high-performing state faster than standard DRL algorithms like TD3 or DDPG.

Conclusion

By treating the video buffer as a controllable bridge between prediction and transmission, QUTA effectively breaks the 360° streaming dilemma. The integration of LTP training ensures that the system remains robust and stable, providing a significantly enhanced viewing experience without requiring detectable changes to playback for the end-user.

Summary updated

BibTeX
@inproceedings{zhang2024deep,
  author = {Guanghui Zhang and Jing Guo},
  title = {{A Deep Reinforcement Learning Approach to Balance Viewport Prediction and Video Transmission in 360° Video Streaming}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
}
Figure