TL;DR
This paper presents the first large-scale study from a network operator's perspective on how cellular network dynamics impact mobile video user engagement. They developed a model that can predict video abandonment with over 87% accuracy using only the first 10 seconds of network traffic data, offering actionable insights for operators to improve user experience.
What Problem Does This Paper Address?
Mobile video streaming is booming, making it crucial for network operators to understand and ensure a good Quality of Experience (QoE) for their users. However, operators typically lack access to the client-side or server-side logs that video providers use to measure user engagement (like video completion or abandonment). Existing methods for analyzing video traffic often rely on deep-packet inspection, which is computationally expensive and raises privacy concerns for large-scale network monitoring. This paper aims to bridge this gap by showing how operators can predict user engagement using readily available network-level data.
Methods
The researchers employed a machine learning approach, specifically decision trees (C4.5) and regression trees (M5P) with bootstrap aggregation (bagging), to model user engagement. This choice was made because these algorithms can handle non-linear relationships and feature interdependence without requiring strict independence assumptions, which are common in other statistical models. They evaluated their models using various feature sets (core network, radio access network, and combined) and at different prediction time windows (full session, first 60 seconds, and first 10 seconds).
Datasets
The study utilized a month-long, anonymized dataset from a major tier-1 cellular network in the United States. This dataset comprised 27 terabytes of video streaming traffic from nearly half a million users. The data was collected from both the Core Network (CN) and the Radio Access Network (RAN), including flow-level information (TCP/IP headers) and radio network statistics. To establish ground truth for user engagement (completion, abandonment, skips), they also collected HTTP-level information, though their final models did not require this for prediction.
Key Insights and Ideas
- Operator-centric view is crucial: Traditional QoE metrics rely on data operators don't have. This work demonstrates a viable alternative using their own data.
- Early prediction is possible: The model can predict user engagement with high accuracy using just the initial 10 seconds of a video session, allowing for proactive interventions.
- Network load and interference are key: Contrary to intuition, higher signal strength (RSCP/RSSI) can correlate with more abandonment, suggesting interference is a bigger issue than coverage in some scenarios. Reducing per-sector load and interference directly improves engagement.
- Handovers can be disruptive: All types of handovers were found to correlate with increased abandonment, highlighting the need for smoother transition mechanisms.
- Throughput isn't everything: While sufficient throughput is necessary, excessively high throughput doesn't always guarantee engagement. There seems to be an optimal range, beyond which abandonment can increase.
- Skips and abandonment have different drivers: The factors influencing video skips are not always the same as those causing abandonment, making it challenging to model both simultaneously.
Key Results
- The developed model can predict video abandonment with over 87% accuracy using only the first 10 seconds of network traffic data.
- Improving the mean signal-to-interference ratio (Ec/Io) by 1 dB reduces video abandonment likelihood by 2%.
- Reducing cell sector load by 10 active users reduces abandonment likelihood by 7%.
- For a 4-way classification (completed/abandoned, skipped/non-skipped), the combined feature set achieved an average accuracy of 75.7% for the full session data.
- For a simplified completed vs. abandoned classification, the combined feature set achieved an average accuracy of 83.5% for the full session data and 77.8% even with only the first 10 seconds of data.
- The M5P regression tree model achieved the best Root Mean Square Error (RMSE) of 0.14 for predicting video download completion using all features and the full session data.
Limitations and Future Work
- Single provider focus: The study's findings are based on data from one video service provider using progressive download. Results might differ for other providers or streaming methods.
- Distinguishing abandonment causes: The model cannot differentiate between abandonment due to network issues versus lack of user interest, as this would require client-side data.
- Future Work: The authors suggest evaluating their model on other network types (e.g., DSL, WiFi) and exploring how the core network features (which are more general) can be leveraged across different infrastructures. They also aim to further refine the models for more granular predictions.
Summary updated