TL;DR
This paper introduces MediSyn, a novel synthetic workload generator for streaming media servers. It addresses the challenge of accurately modeling the complex, long-term, and dynamic access patterns of streaming media services, which differ significantly from traditional web traffic, by proposing new models for file popularity, temporal locality, and content evolution.
What Problem Does This Paper Address?
The increasing popularity of streaming media services and the trend towards shared hosting infrastructures necessitate a deeper understanding of their unique workload characteristics. Traditional web workload generators often assume static content popularity and access patterns, which are insufficient for streaming media. Streaming media involves larger file sizes, longer access durations, and more complex user interactions (like pause/resume), leading to distinct client access patterns and resource demands. Accurately modeling these long-term, dynamic, and non-stationary behaviors is crucial for designing efficient resource allocation mechanisms, capacity planning, and performance evaluation of streaming media servers and content delivery networks.
Methods
The core of this paper is the development of MediSyn, a synthetic workload generator that models streaming media server workloads. The authors introduce a framework that captures several key properties:
- Static Properties:
- File Duration: Modeled using a set of normal distributions to capture "hot spots" in duration distributions, reflecting common media lengths.
- Encoding Bit Rate: Modeled as a discrete distribution, with random matching to file duration due to observed lack of correlation.
- Popularity: Instead of a simple Zipf distribution, they propose the Zipf-Mandelbrot law to better capture the observed "circular" popularity curves in long-term traces. A "k-transformation" is introduced to help fit this model.
- File Access Prefix: Models the portion of a media file a user actually watches, considering both complete and incomplete sessions. Incomplete session prefixes are modeled differently based on file duration (exponential for short files, mixed exponential/uniform for long files).
- Temporal Properties:
- New File Introduction Process: Models how new content is introduced over time, considering daily gaps between introductions and the number of files introduced per day, using Pareto and exponential distributions.
- Life Span: Captures the non-stationary nature of file popularity, where access frequency decreases over time after a file's introduction. Two types are identified: "regular" (lognormal) and "news-like" (Pareto).
- Diurnal Access Pattern: Models the hourly variation in access rates, using a Pareto distribution for interarrival times within an hour and a general diurnal pattern for the entire day.
MediSyn generates these properties for each file and then constructs a sequence of media sessions, which are then sorted globally to create the synthetic trace.
Datasets
The authors used two long-term traces of streaming media services for their analysis and model development:
- HPC (Hewlett-Packard Corporate Media Solutions Server) log: 29 months duration.
- HPL (HPLabs Media Server) log: 21 months duration.
These datasets are used to identify and validate the proposed workload models.
Key Insights and Ideas
- Non-Stationary Popularity is Key: Unlike traditional web workloads, streaming media file popularity is not static. It evolves over time, especially for newly introduced content, making models that capture this dynamism essential.
- Zipf-Mandelbrot Law for Popularity: A simple Zipf distribution is insufficient for long-term streaming media workloads. The Zipf-Mandelbrot law provides a more accurate fit for the observed popularity distributions, which often exhibit a "circular" shape on a log-log scale.
- File Duration "Hot Spots": Media file durations are not uniformly distributed but tend to cluster around specific values (e.g., common video lengths), which can be effectively modeled by a mixture of normal distributions.
- Incomplete Sessions are Prevalent: A significant portion of streaming media sessions are incomplete, with users often only accessing the initial parts of files. The modeling of these incomplete sessions needs to account for file duration.
- Temporal Correlation is Daily, Not Within-Day: While there's temporal correlation in access patterns over longer periods (daily scale), within a single day, access patterns are primarily driven by file popularity. This simplifies the modeling of short-term temporal locality.
Key Results
- MediSyn successfully captures the essential properties of streaming media workloads, as validated against the HPC and HPL traces.
- The proposed models, including Zipf-Mandelbrot for popularity and mixtures of distributions for duration and temporal properties, provide a more accurate representation of streaming media traffic compared to traditional models.
- The paper demonstrates that MediSyn can generate synthetic traces that closely match the original data in terms of various statistical properties. For example, the $\chi^2$ discrepancy measure for duration generation was 0.1140.
Limitations and Future Work
- Client Interactivity: The current version of MediSyn does not explicitly model client interactivities within media sessions (e.g., seeking, fast-forwarding beyond simple pause/resume). The authors plan to extend MediSyn to include these features in future work.
- Workload Analysis Tool: The authors intend to release a workload analysis tool that reflects the property profiles generated by MediSyn, which can aid in tuning the generator's parameters.
- Scalability and Parameterization: While MediSyn is designed to be modular and parameterized, further work might explore its scalability for extremely large-scale deployments and more fine-grained parameter customization for diverse scenarios.
Summary updated