Analyzing Ad Exposure and Content in Child-Oriented Videos on YouTube

Emaan Bilal Khan, Nida Tanveer, Aima Shahid, Mohammad Jaffer Iqbal, Haashim Ali Mirza, Armish Javed, Ihsan Ayyub Qazi, Zafar Ayyub QaziWWW 2024 · OpenReview

TL;DR

This paper presents a large-scale measurement study of YouTube’s advertising ecosystem on child-oriented content, revealing that a child’s safety is heavily dictated by their geographic location. By analyzing over 24,000 ads across ten countries, the authors demonstrate that while strict regional regulations significantly reduce exposure to inappropriate content, massive gaps in YouTube’s "made for kids" labeling and policy enforcement leave millions of children—particularly in the Global South—vulnerable to age-inappropriate and excessively long advertisements.

What Problem Does This Paper Address?

While YouTube maintains a separate "YouTube Kids" app, the primary YouTube platform remains the dominant destination for children under 13. To comply with regulations like COPPA, YouTube uses a "made for kids" (MFK) label to restrict data collection and regulate ads. However, the effectiveness of this system is often questioned.

The authors address three primary gaps:

  1. The Policy Gap: How does ad content differ between countries with high regulatory standards (e.g., the US and UK) and those with low standards (e.g., Bangladesh and Pakistan)?
  2. The Labeling Gap: How much child-oriented content remains unlabeled, and what are the ad-related consequences for those videos?
  3. The Enforcement Gap: To what extent do ads on child-directed videos actually follow YouTube’s own publicly stated duration and content guidelines?

Methods

The researchers employed a multi-step pipeline to collect and analyze data:

  • Automated Scraping: Using a Selenium Webdriver script, they played 1,500 unique videos (750 MFK-labeled and 750 unlabeled child-oriented) across 10 different countries. To avoid personalization bias, they used "clean" logged-out Chrome instances.
  • Cross-Regional Analysis: They selected five "high policy" regions (US, UK, France, Germany, Sweden) and five "low policy" regions (Bangladesh, Sri Lanka, Pakistan, Morocco, Venezuela) based on the Child Online Safety Index.
  • Manual Annotation: A team of coders tagged a subset of 3,000 ads using a deductive coding scheme. Ads were classified as Child-directed, Irrelevant, Inappropriate, or Ambiguous.
  • ML Validation: They used the Google Cloud Video Intelligence API as a benchmark to detect "explicit content" in ads, comparing automated results against their manual annotations.

Datasets

The study utilizes several robust datasets:

  • Labeled Video Set: 750 of the most popular "made for kids" videos identified via Social Blade and the YouTube API.
  • Unlabeled Video Set: 750 popular videos (minimum 400k views) that are clearly child-oriented (e.g., toy unboxings, fairy tales) but lack the MFK label.
  • Ad Dataset: A collection of 24,148 total ads (6,355 unique) captured across the 15,000 total video views (1,500 videos × 10 countries).
  • Tagged Ad Dataset: 3,000 manually annotated video ads, which the authors have shared for future research in inappropriate ad detection.

Key Insights and Ideas

  • Geography is Safety: The most striking insight is that child safety is not a global standard. In low-policy regions, 28.8% of ads were inappropriate, compared to just 3.6% in high-policy regions. This suggests that YouTube’s internal moderation is heavily supplemented (or driven) by external legal pressure.
  • The Non-English Blind Spot: 60.4% of the unlabeled child-oriented dataset consisted of non-English content. This indicates that YouTube’s automated labeling algorithms struggle significantly with regional languages, leaving children in those markets at higher risk.
  • Ad Specs are "Suggestions": YouTube’s 15-second limit for unskippable ads is frequently ignored. In low-policy regions, 77.2% of unskippable ads exceeded this limit, with some "ads" lasting up to 1.8 hours.
  • The "Irrelevant" Majority: Over 75% of ads in high-policy regions were "irrelevant" (e.g., enterprise software ads on a nursery rhyme video). This represents a failure in the ad-matching algorithm that hurts both the advertiser's ROI and the creator's potential earnings.

Key Results

  • Inappropriate Content: Low-policy regions saw an 8x increase in inappropriate ads compared to high-policy regions.
  • Ad Duration: Average ad time per video was 15% longer in low-policy regions (184s vs. 159s).
  • Labeling Impact: Labeling a video "made for kids" significantly reduced ad frequency in low-policy regions (from 1.7 to 1.1 ads per video) but had a negligible effect in high-policy regions (1.9 to 1.8).
  • ML Benchmarking: The Google Video Intelligence API confirmed the manual findings, flagging 76.7% of the "inappropriate" subset as containing potentially adult content.

Limitations and Future Work

The authors acknowledge several limitations:

  • Age Definition: The study focused on the 0-8 age range (aligned with YouTube Kids). Future work could expand to the 9-13 demographic, where "appropriateness" becomes more subjective.
  • Personalization: By using logged-out sessions, the study does not account for how parental browsing history on shared devices might influence the ads shown to children.
  • Platform Scope: The study is limited to YouTube; similar comparative studies are needed for platforms like TikTok and Instagram, which also have massive underage audiences.
  • Future Directions: The authors suggest that YouTube should replicate "YouTube Kids" safeguards (like ad bumpers and disabling outbound links) on the main app for all MFK-labeled content.

Summary updated

Abstract

As a popular choice for video and entertainment streaming, YouTube hosts a large audience, including children, who form a growing proportion of its users. Despite separate “made for kids” labelling and stricter moderation of these videos, inappropriate advertising remains a concern as it threatens the safety of YouTube for young viewers. This paper is the first comparative measurement study that explores how advertisement exposure and content vary across child-oriented videos on YouTube. We do this by conducting a cross-regional advertisement analysis on highly viewed “made for kids” labelled content across a total of five strictly regulated and five unregulated countries. A second front of comparison is carried out between ad patterns on unlabelled and labelled child-oriented videos. Our analysis reveals that the safety of a child’s YouTube experience is shaped significantly by their external environment and surrounding child safety policies. There also exists a gap in YouTube ad and child protection policy enforcement, indicated by the presence of unlabelled child-oriented content with weak ad regulation. We discuss the implications of this inappropriate exposure on children and suggest policy and implementation measures to mitigate this threat.
BibTeX
@inproceedings{khan2024analyzing,
  author = {Emaan Bilal Khan and Nida Tanveer and Aima Shahid and Mohammad Jaffer Iqbal and Haashim Ali Mirza and Armish Javed and Ihsan Ayyub Qazi and Zafar Ayyub Qazi},
  title = {{Analyzing Ad Exposure and Content in Child-Oriented Videos on YouTube}},
  booktitle = {WWW},
  year = {2024},
  url = {https://openreview.net/forum?id=jLi7AmHj9p},
}
Figure