MediSyn

Wenting Tang, Yun Fu, Ludmila Cherkasova, Amin Vahdat

TL;DR

This paper introduces MediSyn, a novel synthetic workload generator for streaming media services. It addresses the need for realistic workload models that capture the unique, dynamic, and non-stationary characteristics of streaming media access patterns, which differ significantly from traditional web traffic. MediSyn's key innovation is a generalized Zipf-like distribution for modeling file popularity and its ability to capture temporal dynamics like file introduction and evolving popularity over time.

What Problem Does This Paper Address?

Modern internet infrastructure, from hosting centers to Content Delivery Networks (CDNs), relies on statistical multiplexing to efficiently serve a multitude of services. To optimize resource allocation and provisioning, especially for emerging services like streaming media, a deep understanding of client access patterns is crucial. Traditional web workload models often assume static content and stable popularity, which are insufficient for streaming media. Streaming media presents unique challenges: larger file sizes, higher bandwidth and computational demands, longer session durations, and importantly, non-stationary file popularity and dynamic content evolution over time. The authors aim to bridge this gap by developing a workload generator that accurately reflects these complex, long-term behaviors.

Methods

MediSyn is built upon two core steps: file property generation and file access generation.

  • File Property Generation: This involves modeling static and temporal properties of media files.
  • Static Properties: MediSyn models file duration using a novel approach of fitting multiple normal distributions to observed "hot spots" in duration data, rather than a single heavy-tail distribution. Encoding bit rate is modeled discretely.
  • Popularity: A key contribution is the generalized Zipf-like distribution. This extends the traditional Zipf distribution by introducing a "k-transformation" to accurately model observed "circular" popularity curves on log-log scales, which are common in long-term traces and not captured by standard Zipf distributions.
  • Temporal Properties: * File Introduction Process: Models how new content is introduced over time, considering daily arrival patterns and the number of files introduced per day.
  • File Life Span: Captures the non-stationary nature of file popularity by modeling how access rates change over time after a file's introduction, using Pareto and lognormal distributions.
  • Diurnal Access Patterns: Models the cyclical nature of access requests throughout a day, allowing for bursty traffic patterns.
  • Prefix Distribution: Accounts for the fact that many clients don't finish streaming media files, modeling incomplete session durations with exponential or heavy-tail distributions depending on file duration.
  • File Access Generation: Once file properties are defined, MediSyn generates sessions by considering the file's introduction time, life span, and diurnal access pattern. Session arrival times are then merged globally.

Datasets

The authors used two long-term streaming media server logs for developing and validating their models:

  • HP Corporate Media Solutions Server (HPC): A 29-month trace covering diverse video content.
  • HPLabs Media Server (HPL): A 21-month trace of HP Labs presentations and seminars.

These datasets are described as representative of real-world streaming media workloads and are used to derive parameters for MediSyn's models.

Key Insights and Ideas

  • Streaming Media is Fundamentally Different: The paper strongly emphasizes that streaming media workloads are not just scaled-up versions of web workloads. Their dynamic, non-stationary nature, especially regarding file popularity and content evolution, requires entirely new modeling approaches.
  • The "Circular" Popularity Curve: The observation that file popularity on a log-log scale often forms a "circular" shape, rather than a straight line (Zipf distribution), is a critical finding. This necessitates a more sophisticated distribution than the standard Zipf.
  • Generalized Zipf-like Distribution: The proposed k-transformation for the Zipf distribution is a clever way to unify the modeling of both traditional Zipfian and the observed circular popularity patterns, providing a more accurate representation.
  • Non-Stationary Popularity is Key: The concept of "file life span" is crucial. It acknowledges that a file's popularity isn't fixed but decays over time, which is a significant departure from static workload models.
  • Client Behavior Matters: The modeling of "prefix" durations, reflecting incomplete playback, is essential for realistic session generation, as it captures common user behavior and resource consumption patterns.

Key Results

  • MediSyn successfully models a range of streaming media workload characteristics, including file duration, encoding bit rate, popularity, new file introduction, life span, and diurnal access patterns.
  • The generalized Zipf-like distribution, through its k-transformation, accurately captures the file popularity distributions of both the HPC and HPL traces, achieving R² values of 0.995.
  • The paper demonstrates that file duration and file popularity, as well as file popularity and encoding bit rate, show negligible correlation in their traces, allowing for random matching of these properties.
  • The study highlights that incomplete session prefixes are often modeled by exponential distributions for shorter files and a combination of exponential and uniform distributions for longer files.
  • New file introduction patterns can be modeled using Pareto and exponential distributions, with time gaps within a day often following a Pareto distribution.

Limitations and Future Work

  • Media Session Interactivity Not Modeled: The current version of MediSyn does not model client interactivity within media sessions (e.g., pause, fast-forward, rewind). This is identified as a key area for future work.
  • Focus on Specific Workloads: While the models are derived from two long-term traces, their generalizability to other types of streaming media services (e.g., live streaming, different content genres) might require further validation.
  • Future Work: The authors explicitly mention enabling MediSyn to support client interactivities within media sessions as a primary future direction. They also suggest that the modular design allows for customization and extension to reflect various scenarios.

Summary updated

Abstract

Currently, Internet hosting centers and content distribution networks leverage statistical multiplexing to meet the performance requirements of a number of competing hosted network services. Developing efficient resource allocation mechanisms for such services requires an understanding of both the short-term and long-term behavior of client access patterns to these competing services. At the same time, streaming media services are becoming increasingly popular, presenting new challenges for designers of shared hosting services. These new challenges result from fundamentally new characteristics of streaming media relative to traditional web objects, principally different client access patterns and significantly larger computational and bandwidth overhead associated with a streaming request. To understand the characteristics of these new workloads we use two long-term traces of streaming media services to develop MediSyn, a publicly available streaming media workload generator. In summary, this paper makes the following contributions: i) we model the long-term behavior of network services capturing the process of file introduction and changing file popularity, ii) we present a novel generalized Zipf-like distribution that captures recently-observed popularity of both web objects and streaming media not captured by existing Zipf-like distributions, and iii) we capture a number of characteristics unique to streaming media services, including file duration, encoding bit rate, session duration and non-stationary popularity of media accesses.
BibTeX
@misc{tang2003medisyn,
  author = {Wenting Tang and Yun Fu and Ludmila Cherkasova and Amin Vahdat},
  title = {{MediSyn}},
  year = {2003},
  doi = {10.1145/776325.776327},
}
Figure