Enhancing Neural Adaptive Wireless Video Streaming via Cross-Layer Information Exposure and Online Tuning

Lingzhi Zhao, Ying Cui, Yuhang Jia, Yunfei Zhang, Klara Nahrstedt

TL;DR

This paper introduces eA3C, an enhanced Deep Reinforcement Learning (DRL) framework for adaptive video streaming that outperforms state-of-the-art methods like Pensieve. By exposing lower-layer network information (like MAC rates) and implementing a novel "online tuning" mechanism, the system can predict network fluctuations 200–300ms faster than traditional methods and tailor its behavior to a specific user's real-time environment.

What Problem Does This Paper Address?

Adaptive Bitrate (ABR) algorithms are the "brains" behind video streaming, deciding which quality level to download to prevent buffering while maximizing resolution. Most current DRL-based ABR algorithms suffer from three main flaws:

  1. Lagging Indicators: They rely solely on Application (APP) layer information (like throughput and buffer levels). However, the APP layer is the last to know about network changes; lower layers (MAC/Physical) feel the "jitter" much sooner.
  2. Heuristic Training: Standard Actor-Critic (A3C) methods train the policy (actor) and value (critic) networks separately or alternatively, which lacks rigorous convergence guarantees.
  3. The "Average User" Trap: Models are typically trained offline on massive datasets. When deployed, they often perform poorly for a specific user whose unique mobility or signal environment doesn't match the "average" training data.

Methods

The authors propose a multi-pronged approach to solve these issues:

  • Cross-Layer Information Exposure: The model incorporates lower-layer metrics—specifically MAC rate, Physical Resource Blocks (PRBs), and Modulation and Coding Scheme (MCS)—into the DRL state. This allows the agent to "see" network drops before they impact the video buffer.
  • Enhanced A3C (eA3C): Unlike standard A3C, eA3C uses a joint optimization formulation. It treats the policy and value functions as a unified optimization problem, leading to faster convergence and better performance. The architecture uses 1D Convolutional Neural Networks (CNNs) to extract temporal features from both APP and lower-layer data.
  • Continual Learning (Online Tuning): To bridge the gap between offline training and real-world deployment, the authors introduce two tuning methods:
  • eA3C-OTP: Tunes only the policy network for the specific user.
  • eA3C-OTPV: Jointly tunes both policy and value networks. These methods add "tunable components" to the pre-trained offline model, allowing it to adapt to a user's specific signal conditions in real-time without starting from scratch.

Datasets

The researchers collected five distinct datasets containing both APP-layer throughput and lower-layer quantities (MAC rate, PRB, MCS).

  • The data was split into Set-OFF (for offline training/testing) and Set-ON (for online tuning evaluation).
  • These datasets reflect various network environments, from "strong" to "weak" signal conditions, providing a more granular look at how lower-layer info helps in challenging scenarios.

Key Insights and Ideas

  • The 300ms Advantage: The authors demonstrate that MAC-layer rates respond to environment changes 200–300 milliseconds faster than APP-layer throughput. In the world of high-speed streaming, this is an eternity that allows the ABR to downshift quality before a buffer stall occurs.
  • Joint vs. Alternating Optimization: Jointly optimizing the actor and critic captures the intrinsic relationship between the policy and the value function better than the standard "heuristic" alternating method used in previous work.
  • Adaptation is Key for Weak Signals: Online tuning provides the biggest gains in "weak" network environments. This suggests that while a general model works "well enough" for good signals, personalized tuning is essential for users on the edge of coverage.

Key Results

  • Offline Gains: The proposed eA3C policy improved Quality of Experience (QoE) by 6.8% to 14.4% compared to state-of-the-art baselines (like Pensieve and MPC) in offline scenarios.
  • Online Gains: When online tuning was applied, the system achieved an additional 6% to 28% gain in QoE over the already-optimized offline policy.
  • Efficiency: Despite the added complexity of lower-layer data, the neural network inference remains fast enough for real-time use, significantly outperforming exhaustive search methods like MPC in terms of computation time.

Limitations and Future Work

  • Information Access: The primary limitation is the availability of lower-layer information. In many mobile operating systems, MAC-layer data is restricted or requires specific APIs that might not be accessible to standard third-party apps.
  • Computational Overhead: While inference is fast, online tuning requires additional CPU/GPU cycles on the client device, which could impact battery life.
  • Future Directions: The authors suggest exploring more complex "continual learning" strategies to ensure the model doesn't "forget" general network patterns while over-fitting to a specific local environment.

Summary updated

Abstract

Deep reinforcement learning (DRL) demonstrates its promising potential in adaptive video streaming and has recently received increasing attention. However, existing DRL-based methods for adaptive video streaming mainly use application (APP) layer information, adopt heuristic training methods, and are not robust against continuous network fluctuations. This paper aims to boost the quality of experience (QoE) of adaptive wireless video streaming by using cross-layer information, deriving a rigorous training method, and adopting effective online tuning methods with real-time data. First, we formulate a more comprehensive and accurate adaptive wireless video streaming problem as an infinite stage discounted Markov decision process (MDP) problem by additionally incorporating past and lower-layer information. This formulation allows a flexible tradeoff between QoE and computational and memory costs for solving the problem. In the offline scenario (only with pre-collected data), we propose an enhanced asynchronous advantage actor-critic (eA3C) method by jointly optimizing the parameters of parameterized policy and value function. Specifically, we build an eA3C network consisting of a policy network and a value network that can utilize cross-layer, past, and current information and jointly train the eA3C network using pre-collected samples. In the online scenario (with additional real-time data), we propose two continual learning-based online tuning methods for designing better policies for a specific user with different QoE and training time tradeoffs. The proposed online tuning methods are robust against continuous network fluctuations and more general and flexible than the existing online tuning methods. Finally, experimental results show that the proposed offline policy can improve the QoE by 6.8% to 14.4% compared to the state-of-the-arts in the offline scenario, and the proposed online policies can achieve $6.3\%$ to 55.8% gains in QoE over the state-of-the-arts in the online scenario.
BibTeX
@article{zhao2025enhancing,
  author = {Lingzhi Zhao and Ying Cui and Yuhang Jia and Yunfei Zhang and Klara Nahrstedt},
  title = {{Enhancing Neural Adaptive Wireless Video Streaming via Cross-Layer Information Exposure and Online Tuning}},
  journal = {IEEE Trans. Multimedia},
  year = {2025},
  url = {https://openreview.net/forum?id=w3hvJ6Bw2s},
}

Loading related content...

Figure