TL;DR
This paper introduces MediSyn, a novel synthetic workload generator for streaming media services. It addresses the need for realistic workload models that capture the unique, dynamic, and non-stationary characteristics of streaming media access patterns, which differ significantly from traditional web traffic. MediSyn's key innovation is a generalized Zipf-like distribution for modeling file popularity and its ability to capture temporal dynamics like file introduction and evolving popularity over time.
What Problem Does This Paper Address?
Modern internet infrastructure, from hosting centers to Content Delivery Networks (CDNs), relies on statistical multiplexing to efficiently serve a multitude of services. To optimize resource allocation and provisioning, especially for emerging services like streaming media, a deep understanding of client access patterns is crucial. Traditional web workload models often assume static content and stable popularity, which are insufficient for streaming media. Streaming media presents unique challenges: larger file sizes, higher bandwidth and computational demands, longer session durations, and importantly, non-stationary file popularity and dynamic content evolution over time. The authors aim to bridge this gap by developing a workload generator that accurately reflects these complex, long-term behaviors.
Methods
MediSyn is built upon two core steps: file property generation and file access generation.
- File Property Generation: This involves modeling static and temporal properties of media files.
- Static Properties: MediSyn models file duration using a novel approach of fitting multiple normal distributions to observed "hot spots" in duration data, rather than a single heavy-tail distribution. Encoding bit rate is modeled discretely.
- Popularity: A key contribution is the generalized Zipf-like distribution. This extends the traditional Zipf distribution by introducing a "k-transformation" to accurately model observed "circular" popularity curves on log-log scales, which are common in long-term traces and not captured by standard Zipf distributions.
- Temporal Properties: * File Introduction Process: Models how new content is introduced over time, considering daily arrival patterns and the number of files introduced per day.
- File Life Span: Captures the non-stationary nature of file popularity by modeling how access rates change over time after a file's introduction, using Pareto and lognormal distributions.
- Diurnal Access Patterns: Models the cyclical nature of access requests throughout a day, allowing for bursty traffic patterns.
- Prefix Distribution: Accounts for the fact that many clients don't finish streaming media files, modeling incomplete session durations with exponential or heavy-tail distributions depending on file duration.
- File Access Generation: Once file properties are defined, MediSyn generates sessions by considering the file's introduction time, life span, and diurnal access pattern. Session arrival times are then merged globally.
Datasets
The authors used two long-term streaming media server logs for developing and validating their models:
- HP Corporate Media Solutions Server (HPC): A 29-month trace covering diverse video content.
- HPLabs Media Server (HPL): A 21-month trace of HP Labs presentations and seminars.
These datasets are described as representative of real-world streaming media workloads and are used to derive parameters for MediSyn's models.
Key Insights and Ideas
- Streaming Media is Fundamentally Different: The paper strongly emphasizes that streaming media workloads are not just scaled-up versions of web workloads. Their dynamic, non-stationary nature, especially regarding file popularity and content evolution, requires entirely new modeling approaches.
- The "Circular" Popularity Curve: The observation that file popularity on a log-log scale often forms a "circular" shape, rather than a straight line (Zipf distribution), is a critical finding. This necessitates a more sophisticated distribution than the standard Zipf.
- Generalized Zipf-like Distribution: The proposed k-transformation for the Zipf distribution is a clever way to unify the modeling of both traditional Zipfian and the observed circular popularity patterns, providing a more accurate representation.
- Non-Stationary Popularity is Key: The concept of "file life span" is crucial. It acknowledges that a file's popularity isn't fixed but decays over time, which is a significant departure from static workload models.
- Client Behavior Matters: The modeling of "prefix" durations, reflecting incomplete playback, is essential for realistic session generation, as it captures common user behavior and resource consumption patterns.
Key Results
- MediSyn successfully models a range of streaming media workload characteristics, including file duration, encoding bit rate, popularity, new file introduction, life span, and diurnal access patterns.
- The generalized Zipf-like distribution, through its k-transformation, accurately captures the file popularity distributions of both the HPC and HPL traces, achieving R² values of 0.995.
- The paper demonstrates that file duration and file popularity, as well as file popularity and encoding bit rate, show negligible correlation in their traces, allowing for random matching of these properties.
- The study highlights that incomplete session prefixes are often modeled by exponential distributions for shorter files and a combination of exponential and uniform distributions for longer files.
- New file introduction patterns can be modeled using Pareto and exponential distributions, with time gaps within a day often following a Pareto distribution.
Limitations and Future Work
- Media Session Interactivity Not Modeled: The current version of MediSyn does not model client interactivity within media sessions (e.g., pause, fast-forward, rewind). This is identified as a key area for future work.
- Focus on Specific Workloads: While the models are derived from two long-term traces, their generalizability to other types of streaming media services (e.g., live streaming, different content genres) might require further validation.
- Future Work: The authors explicitly mention enabling MediSyn to support client interactivities within media sessions as a primary future direction. They also suggest that the modular design allows for customization and extension to reflect various scenarios.
Summary updated