Understanding the effectiveness of video ads: a measurement study

S. Shunmuga Krishnan, Ramesh K. SitaramanInternet Measurement Conference 2013 · DOI

TL;DR

This paper presents the first rigorous scientific study on the effectiveness of online video ads, moving beyond simple correlations to establish causal rules. By analyzing a massive dataset of video ad impressions, the authors reveal how ad placement, length, video content, and viewer characteristics influence ad completion rates, offering crucial insights for monetizing online video.

What Problem Does This Paper Address?

The explosive growth of online video has created a significant challenge for content providers: how to effectively monetize this content. A prevalent model is in-stream advertising, where ads play before, during, or after the video. However, understanding what makes these ads "effective" – specifically, whether viewers watch them to completion – has been largely based on intuition rather than scientific evidence. This paper aims to fill that gap by scientifically investigating the factors that determine video ad effectiveness, measured primarily by ad completion and abandonment rates, and deriving causal rules for viewer behavior.

Methods

The authors employ a novel approach combining correlational analysis with Quasi-Experimental Designs (QEDs). While correlational analysis helps identify associations (e.g., mid-roll ads have higher completion rates), QEDs are used to establish causality. The core QED technique involves matched-pair analysis, where views with different ad treatments (e.g., mid-roll vs. pre-roll) are matched based on confounding variables like viewer identity, video content, and connection type. By comparing outcomes within these matched pairs, the researchers can isolate the causal impact of the specific factor being studied. They also utilize Information Gain Ratio (IGR) to quantify the influence of different factors on ad completion.

Datasets

The study utilizes a large-scale dataset collected from Akamai's video delivery network, comprising approximately 65 million unique viewers, 362 million videos, and 257 million ad impressions from 33 video providers worldwide over a 15-day period in April 2013. The data includes detailed metrics on video playback, ad impressions, and viewer behavior.

Key Insights and Ideas

  • Ad Placement is Crucial: The position of an ad within a video significantly impacts its completion rate. Mid-roll ads, appearing when the viewer is already engaged with the content, are far more effective than pre-roll or post-roll ads.
  • Shorter Ads are Generally Better: Counterintuitively, longer ads (20-30 seconds) tend to have lower completion rates than shorter ones (15 seconds), suggesting viewer patience wanes with ad duration.
  • Content Matters, But Context Even More: While ad and video content are important, the context in which an ad is placed (e.g., within a long-form movie vs. a short news clip) has a more pronounced effect on completion rates. Viewers are more willing to tolerate ads when they perceive greater value from the associated content.
  • Repeat Visitors are More Engaged: Viewers who frequently visit a video provider's site exhibit higher ad completion rates, likely due to a greater interest in the site's content.
  • Patience for Ads vs. Video Startup: Viewers are significantly more patient when waiting for an ad to complete than when waiting for a slow-loading video to start. This difference is attributed to the expected and predictable nature of ad waits versus the unexpected and uncertain duration of video loading delays.

Key Results

  • Ad Position: An ad is 18.1% more likely to complete when placed as a mid-roll compared to a pre-roll, and 14.3% more likely when placed as pre-roll compared to a post-roll.
  • Ad Length: While initial correlations suggested 30-second ads were most effective, QED analysis revealed that shorter ads are more likely to complete. Specifically, a 15-second ad is 2.86% more likely to complete than a 20-second ad, and a 20-second ad is 3.89% more likely than a 30-second ad.
  • Video Length: Ads placed in long-form videos (e.g., movies, TV episodes) are 4.2% more likely to complete than the same ads placed in short-form videos (e.g., news clips).
  • Viewer Behavior: Repeat visitors have an 84.9% ad completion rate, compared to 78% for one-time visitors.
  • Abandonment Timing: Approximately one-third of viewers who abandon an ad do so in the first quarter, while about two-thirds leave by the halfway point.
  • Patience Comparison: At the 10-second mark, viewers waiting for a slow-loading video abandon at a rate more than three times higher than those watching a pre-roll ad (45.8% vs. 13.4%).

Limitations and Future Work

The authors acknowledge that while QEDs provide strong evidence for causal relationships, they cannot offer definitive proof of causality, as unmeasured confounding variables could still exist. Future work could explore other metrics beyond completion and abandonment rates, such as click-through rates (CTR) and post-ad engagement. Additionally, developing sophisticated ad placement algorithms that leverage these insights to optimize ad delivery for both advertisers and content providers is a promising avenue. The study also highlights the need for further research into how viewer demographics and psychographics interact with ad effectiveness.

Summary updated

BibTeX
@inproceedings{krishnan2013understanding,
  author = {S. Shunmuga Krishnan and Ramesh K. Sitaraman},
  title = {{Understanding the effectiveness of video ads: a measurement study}},
  booktitle = {Internet Measurement Conference},
  year = {2013},
  doi = {10.1145/2504730.2504748},
}

Loading related content...

Figure