On-Demand Video Dispatch Networks: A Scalable End-to-End Learning Approach

Damao Yang, Sihan Peng, He Huang, Hongliang XuearXiv 2019 · arXiv

TL;DR

Researchers from Bilibili, NYU, Purdue, and Cornell have developed a scalable, end-to-end learning system that predicts "hot" videos and dispatches them to Content Delivery Networks (CDNs) during off-peak hours. By combining video clustering via autoencoders with a dual CNN-RNN temporal architecture, they achieved a 17% prediction accuracy—a nearly 6x improvement over traditional baseline methods.

The Challenge of Video-on-Demand Dispatch

In the world of Video-on-Demand (VOD), the "peak hour" is the primary bottleneck. When millions of users request the same trending videos simultaneously, it puts immense strain on the Master Server and the core network. To mitigate this, providers use CDNs—servers located closer to users—to cache popular content.

Overview of the video dispatch system architecture.

However, dispatching videos to CDNs is a massive optimization challenge. With over 50 million videos in a library like Bilibili’s but limited storage and bandwidth at each CDN node, providers cannot mirror everything. Traditional "reactive" methods only move a video to a CDN after it hits a certain popularity threshold, which often results in "CDN misses" during critical peak times. The goal is to move from reactive to proactive dispatching at scale.

Proposed Methodology: End-to-End Learning

The researchers propose a system composed of two coupled neural networks that share "temporal layers" to understand how video popularity changes over time.

The dual-network architecture combining clustering and policy learning.

1. Clustering Network

To handle the "curse of dimensionality" (millions of videos), the system first groups videos with similar request patterns. It uses an autoencoder to map video request histories onto a 2D plane, which is then divided into hierarchical blocks. This reduces the problem from managing millions of individual videos to managing a fixed number of clusters, making the optimization computationally tractable.

2. Policy Network

This network ranks these clusters to determine dispatch probabilities. It utilizes a unique Temporal Layer architecture:

  • CNN Module: Handles "intraday" patterns. It features a Peak Convolutional Part (PCP) to identify spikes and a Mean Convolutional Part (MCP) to track average trajectories.
  • RNN Module (GRU): Handles "interday" dynamics, capturing how popularity evolves over several days.

3. Joint Training

Unlike systems where clustering and prediction are separate, these networks are trained together. They use an asynchronous training mechanism where the clustering model and the temporal/policy models update each other iteratively, ensuring the clusters remain optimized for the final dispatch goal.

Experimental Results

The study utilized a massive real-world dataset from Bilibili Inc., encompassing 50 million videos and 50 CDNs over 15 consecutive days.

  • Accuracy: The system achieved an average prediction accuracy of 17%, compared to just 3% for the existing frequency-threshold baseline.
  • Efficiency: The model successfully managed the dispatch of 10,000 videos to 50 CDNs daily, taking about 4 hours to compute on a 100-machine cluster.
  • Stability: The hierarchical block method for clustering ensured consistent indices, which is vital for the policy network to learn effectively over time compared to standard K-means.

Conclusion and Future Work

The proposed approach demonstrates that end-to-end learning combined with strategic clustering can significantly improve CDN efficiency at scale. While highly effective at predicting patterns based on history, the authors acknowledge that the system may still struggle with sudden, unpredictable "viral" events. Future work will focus on incorporating more metadata (like video tags or holiday schedules) and developing robust risk controls to handle rapid shifts in user behavior.

Summary updated

BibTeX
@misc{yang2019demand,
  author = {Damao Yang and Sihan Peng and He Huang and Hongliang Xue},
  title = {{On-Demand Video Dispatch Networks: A Scalable End-to-End Learning Approach}},
  year = {2019},
  eprint = {1901.04295},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1901.04295},
}
Figure