Abstract
Adaptive bitrate (ABR) streaming is the standard for delivering smooth video under fluctuating network conditions. However, most current ABR algorithms are content-agnostic, ignoring the semantic importance of video segments. This paper introduces a Content-of-Interest (CoI) based rate adaptation scheme. By utilizing deep learning to identify "interesting" video segments and a Deep Q-Network (DQN) to prioritize their quality, the system improves subjective viewer experience without sacrificing traditional network performance metrics.
Introduction
Adaptive Bitrate (ABR) streaming, used by platforms like YouTube and Netflix, switches quality levels to prevent buffering. A significant limitation of existing ABR logic is that it treats all content equally; a high-action goal in a soccer match is given the same priority as a static shot of the crowd. From a user's perspective, pixelation during a critical moment is far more disruptive than during a mundane one. The authors argue that "interestingness"—the level of subjective engagement a segment draws—should dictate the allocation of the limited "bitrate budget."
System Framework
The proposed framework operates in two main stages: offline semantic analysis and online rate adaptation. The interaction between the server-side content analysis and the client-side player is central to this approach.
- Interestingness Recognition: Using 3D Convolutional Networks (3D ConvNets), the system analyzes spatiotemporal features (movement over time) rather than just static frames. The model extracts features from video chunks and outputs a real-valued "interestingness score."
- DQN-Based Rate Adaptation: Rate adaptation is formulated as a Reinforcement Learning (RL) problem where a Deep Q-Network (DQN) agent selects the optimal bitrate for the next chunk.
DQN-Based Rate Adaptation
The DQN agent is trained to make decisions based on a multi-dimensional state that incorporates both network conditions and content semantics.
As shown in the architecture, the agent considers predicted bandwidth, current buffer occupancy, the previous bitrate, and the interestingness scores of upcoming chunks. The "secret sauce" is a modified reward function that weights video quality by its interestingness score. If the agent delivers high quality during an "interesting" segment, it receives a significantly higher reward, teaching it to "save" bandwidth for high-stakes moments.
Key Results and Insights
The system was evaluated using the TVSum dataset for interestingness and realistic bandwidth traces (FCC and 3G/HSDPA) for network simulation. Key findings include:
- High Precision: The interestingness recognition model achieved a Mean Squared Error (MSE) of 0.02, accurately reflecting human engagement levels.
- Semantic Alignment: Unlike standard methods (BBA or Robust-MPC), the CoI method showed a strong positive correlation between content interestingness and allocated bitrate.
- No Technical Penalty: The system achieved content-awareness without a significant penalty to objective metrics like rebuffering time or bitrate variation. In fact, the CoI method reached the highest mean average bitrate per session among all tested baselines.
Conclusion and Future Work
By treating interestingness as a multiplier for quality in the RL reward function, the agent naturally learns to prioritize critical content. While the current model uses an average human interest score, future work could involve personalizing adaptation to individual preferences and optimizing the DQN for low-power mobile devices.
Summary updated